If you're accepted, you'll receive $500,000 in investment, plus access to the best startup community in the world.

如果您被錄取，您將獲得 500,000 美元的投資，還可以進入世界上最好的初創企業社區。

So apply now and come build the future with us.

現在就申請吧，和我們一起創造未來。

If you ask people what AGI was, they would say it's a model that you can actually interact with.

如果你問人們什麼是 AGI，他們會說這是一個可以真正與之互動的模型。

It passes the Turing test.

它通過了圖靈測試。

It can look at things.

它可以觀察事物。

It can write code.

它可以編寫代碼。

It can even draw an image for you.

它甚至可以為你繪製影像。

Way there.

去那裡的路

Yeah.

是啊

And like we've had this for years.

就像我們多年來都是這樣。

And if you said, okay, well, what happens when you get all those capabilities?

如果你說，好吧，那麼當你獲得所有這些能力時會發生什麼呢？

Say, well, everybody's out of a job and game over for humanity.

說，好吧，每個人都失業了，人類的遊戲結束了。

And none of that is happening.

而這些都沒有發生。

I think in the big picture, we're reaching that bottleneck for pre-training and data.

我認為，從大局來看，我們正在達到預培訓和數據的瓶頸。

But now we have this new mechanism with reasoning and test time compute.

但現在，我們有了這種新的機制，可以進行推理和測試時間計算。

What we're going to see out of reasoning is that it's really going to unlock the possibility of agents to do actions on your behalf, which has sort of always been possible, but it's just never been quite good enough.

我們將從推理中看到的是，它將真正釋放代理人代表你採取行動的可能性。

You really need a lot of reliability.

你真的需要很多可靠性。

I think that is now in sight.

我想這已經指日可待了。

Hey guys, we have a real treat today.

嘿，夥計們，今天有好戲看了。

Bob McGrew, formerly chief research officer at OpenAI.

Bob McGrew，OpenAI 前首席研究官。

You were a part of building a lot of the research team.

你參與組建了很多研究團隊。

What was that like early at OpenAI?

早期在 OpenAI 工作是什麼感覺？

The really interesting thing about OpenAI is that I did not originally intend to go to a research lab.

OpenAI 真正有趣的地方在於，我原本並不打算去研究實驗室。

When I left Palantir, I wanted to start a company.

離開 Palantir 時，我想創辦一家公司。

I had a thesis that robotics would be the first real business that was built out of deep learning.

我曾有過這樣一個論斷，即機器人技術將是第一個通過深度學習建立起來的真正業務。

This was back in 2015.

這要追溯到 2015 年。

And I talked my way into a friend's nonprofit.

我還說服了一個朋友加入了非營利組織。

I never had a badge, but I would go in, he'd open the door for me.

我沒有徽章，但我進去後，他會給我開門。

And I learned deep learning by teaching a robot how to play checkers from vision.

我還通過教機器人如何從視覺出發下跳棋來學習深度學習。

And in the process of doing this, I learned a lot about robotics.

在這個過程中，我學到了很多關於機器人的知識。

And I learned that robotics was definitely not the right startup to start in 2015 or 2016.

我還了解到，機器人技術絕對不是 2015 年或 2016 年創業的正確選擇。

I ended up going to OpenAI basically because it was a place full of very smart people and it had big ambitions.

我最終去了 OpenAI，基本上是因為這裡有很多非常聰明的人，而且有很大的抱負。

It was a place where I could really learn.

在這裡，我可以真正學到東西。

I had all this management experience from Palantir, but it was just a place for me to really become an expert in deep learning.

我在 Palantir 積累了豐富的管理經驗，但那只是讓我真正成為深度學習專家的地方。

And from there, figure out what it could actually be used and applied for.

然後，再找出它的實際用途和適用範圍。

What were some of the earliest things that you remember working on?

在您的記憶中，最早的工作是什麼？

And how did that play into what everyone knows OpenAI to be now?

這又是如何影響大家對 OpenAI 的認識的？

Yeah.

是啊

When OpenAI started, the goal was always to build AGI.

OpenAI 創立之初，目標始終是構建 AGI。

But the theory early on was that we would build AGI by doing a lot of research and writing a lot of papers.

但早期的理論是，我們將通過大量研究和撰寫大量論文來構建 AGI。

And we knew that this was a bad theory.

我們知道這是一個錯誤的理論。

I think for a lot of the early people who were startup people, Sam, Greg, myself, it felt painful and a little academic.

我認為，對於很多早期的創業者來說，比如山姆、格雷格和我自己，這感覺很痛苦，而且有點學術化。

But at the same time, it was what we could do at the time.

但同時，這也是我們當時所能做的。

And so some of the early projects, I worked on a robotics project where we took a robot hand, a humanoid robot hand, and we taught it to solve Rubik's Cube.

是以，在早期的一些項目中，我參與了一個機器人項目，我們用一隻機器手，一隻仿人機器手，教它解魔術方塊。

The idea in doing that was that if we could make the environments complicated enough, the artificial intelligence would be able to generalize out of the narrow domain it was taught and learn something more complicated, which was one of the ideas that later we see coming back with LLMs.

這樣做的想法是，如果我們能把環境變得足夠複雜，人工智能就能跳出狹隘的教學領域，學習更復雜的東西。

The other really early big project was solving Dota 2.

另一個真正的早期大項目是解決 Dota 2。

So there's a long history of solving games as a path towards building better AI, from Othello to Go.

是以，從黑白棋到圍棋，將解決遊戲問題作為構建更好人工智能的途徑由來已久。

And after beating Go, the next hardest set of games are actually video games.

而在戰勝圍棋之後，下一組最難的遊戲其實是電子遊戲。

They're not very classy, but they're a lot of fun.

它們並不高雅，但非常有趣。

And I can assure you that mathematically, they were harder.

我可以向你保證，從數學上講，它們更難。

And so DeepMind went after StarCraft, OpenAI went after Dota 2.

於是，DeepMind 開始追逐《星際爭霸》，OpenAI 開始追逐《Dota 2》。

And there was real insight that was generated there, which was that it really strengthened our belief that scale was the path to improving artificial intelligence.

我們從中獲得了真知灼見，這讓我們更加堅信，規模化是改進人工智能的必由之路。

That with Dota 2, the secret idea was that we could take huge amounts of experience and feed it into a neural network, and that the neural network would actually learn and generalize from that.

在《Dota 2》中，我們的祕密想法是，我們可以將大量的經驗輸入到神經網絡中，然後神經網絡就能從中學習和歸納。

And later, we actually went back and applied this to the robot hand, and that became the key idea for the robot hand.

後來，我們又把它應用到了機械手上，這也成為了機械手的關鍵創意。

And at the same time as these two big projects were going on, Alec Radford was experimenting with language.

在這兩個大項目進行的同時，亞歷克-拉德福德也在進行語言實驗。

And the core idea behind GPT-1 is that if you have a transformer, and you apply this super simple objective of guessing the next token, guessing the next word, that that would be enough signal that you could actually have something that would be able to generate coherent text.

GPT-1 背後的核心理念是，如果你有一個變壓器，並應用這個超級簡單的目標來猜測下一個標記，猜測下一個單詞，這將是足夠的信號，你實際上可以有一個能夠生成連貫文本的東西。

And in retrospect, it sounds sort of obvious, right?

現在回想起來，這聽起來很明顯，對嗎？

Like, you know, clearly this was going to work, but no one thought this would work at the time.

就像，你知道，這顯然是行得通的，但當時沒有人認為這行得通。

Alec really had to persevere for years in order to make this work, and that became GPT-1.

艾力克真的是堅持了好幾年才把這件事做成的，這就是後來的 GPT-1。

And then after GPT-1 seemed successful, we brought in the ideas from Dota and from the robot hand of training at larger and larger amounts of scale, and training on a really diverse set of data, and looking for generalization.

在 GPT-1 似乎取得成功後，我們引入了 Dota 和機器人手的理念，即以越來越大的規模進行訓練，在真正多樣化的數據集上進行訓練，並尋求泛化。

And together, that brings you to GPT-2, and GPT-3, and GPT-4.

合起來就是 GPT-2、GPT-3 和 GPT-4。

So one of the things that OpenAI really pioneered and sort of figured out was this concept of scale.

是以，OpenAI 真正開創並摸索出的一個概念就是 "規模"。

How is it that it was OpenAI that made the right decisions and sort of found large language models first?

為什麼是 OpenAI 做出了正確的決定，並首先找到了大型語言模型？

Early on, there were sort of a couple big projects, as I said, and then some room for exploratory research.

正如我所說，早期有幾個大項目，然後還有一些探索性研究的空間。

And at the very earliest days, the exploratory research was really about what the researcher wanted to do, but also it was about sort of the company's opinion.

在最開始的時候，探索性研究實際上是關於研究人員想要做什麼，同時也是關於公司的意見。

And in this, it was primarily formed by Ilya, with influence from a lot of people, but I think Ilya was really the guiding light here early on.

在這一點上，它主要是由伊利亞形成的，也受到了很多人的影響，但我認為伊利亞在早期確實是這裡的指路明燈。

Sometimes I think about the OpenAI culture, and I like to oppose it to sort of Google Brain and to DeepMind.

有時我在思考 OpenAI 文化，我喜歡把它與谷歌大腦和 DeepMind 對立起來。

And so early on, the DeepMind culture was a caricature.

是以，在早期，DeepMind 的文化就是一幅漫畫。

Demis had a big plan, and he wanted to hire a bunch of researchers so he could tell them to move forward with his plan.

戴米斯有一個大計劃，他想僱一批研究人員，這樣他就可以告訴他們推進他的計劃。

And Google Brain said, let's rebuild academia.

谷歌大腦說，讓我們重建學術界吧。

Let's like bring in all these super talented researchers.

讓我們把這些才華橫溢的研究人員都請來吧。

Let's not tell them anything.

我們什麼都不要告訴他們。

Let's just let them figure out what they want to do, give them lots of resources and hope that amazing products pop out.

讓他們自己想辦法吧，給他們提供大量資源，希望他們能創造出令人驚歎的產品。

And of course they did, but they didn't necessarily happen at Google.

當然，它們確實發生了，但不一定發生在谷歌。

And we took a different approach, which was really more like a startup, where there was no sort of big centralized plan, but at the same time, people didn't have, it wasn't just sort of, let's let a thousand flowers bloom.

我們採取了一種不同的方法，這種方法其實更像一家初創公司，沒有什麼集中式的大計劃，但與此同時，人們也沒有那種 "百花齊放 "的想法。

Instead, we had opinions about what needed to be done and things like, how do you show scale as a way of making your idea get better?

相反，我們對需要做的事情有自己的看法，比如，如何顯示規模才能讓你的想法變得更好？

And that opinion was set by the research leadership.

而這一意見是由研究領導層確定的。

Again, early on, people like Ilya, people like Dario, that was how we made sure that we didn't just sort of throw resources at everybody, but neither did we have just one set of ideas that were there.

同樣，在早期，像伊利亞這樣的人，像達里奧這樣的人，就是我們如何確保我們不只是把資源扔給每個人，但我們也不是隻有一套想法。

We found this sort of happy medium between the two.

我們在二者之間找到了一種平衡點。

I guess one of the critiques of maybe pure academia or some of the AI research labs, we don't have to name any of them, but we've heard stories about looking at the number of researchers on any given paper, there might be way more people on it.

我想，對純粹的學術界或一些人工智能研究實驗室的責備之一是，我們不必點名責備它們，但我們聽說過這樣的故事：從某篇論文的研究人員數量來看，可能有更多的人参與其中。

And if you really dig into some of the papers there, they look like maybe a little bit of this plus a little bit of that.

如果你仔細研究其中的一些論文，就會發現它們看起來就像是這個加一點那個。

And that sort of reflected the nature of that's what it took to get compute.

這也反映了計算的本質。

And this is at other AI labs.

這也是其他人工智能實驗室的情況。

I mean, what was it about open AI where you were able to sort of avoid that?

我的意思是，在開放式人工智能中，你是如何避免這種情況的？

Well, I think the paper example is a really interesting example, because I think that's sort of both good and bad.

我認為紙張的例子非常有趣，因為我認為這既有好處也有壞處。

I am hugely positive on academics and researchers, but actually pretty negative on academia.

我對學者和研究人員的評價非常積極，但實際上對學術界的評價相當消極。

I think academia is good for this very narrow thing of small groups, trying out crazy ideas, but academia has a lot of incentives that prevent people from collaborating.

我認為，學術界對於小團體、嘗試瘋狂想法這種非常狹隘的事情來說是件好事，但學術界有很多阻礙人們合作的激勵機制。

And in particular in academia, there's this obsession with credit.

特別是在學術界，人們對學分有一種痴迷。

One of the things that's interesting about the way that papers have turned out in big labs is that early on we made the decision that we would try to be as Catholic as possible in putting everybody's name on it.

關於大型實驗室的論文成果，其中一件有趣的事情是，我們很早就做出決定，我們將盡可能地把每個人的名字都寫在論文上。

And one of the early robotics papers, we actually said cite as open AI because we didn't wanna get into a fight.

在早期的一篇機器人論文中，我們實際上說引用的是開放式人工智能，因為我們不想捲入爭論。

The first author is the one who gets cited and their name shows up every single time.

第一作者是被引用的人，他們的名字每次都會出現。

So we said, we're not gonna try to have this fight.

所以我們說，我們不打算打這場架了。

We're not gonna say who is the person who really did it.

我們不會說誰是真正的凶手。

We're just gonna say cite as open AI.

我們只能說是開放式人工智能。

And I think that is actually a really important cultural piece, the ability to accept that people want credit, but to be able to channel it into, it's your internal reputation, not the position you have on the paper that really matters.

我認為，這實際上是一個非常重要的文化部分，即接受人們想要榮譽的能力，但能夠將其轉化為你的內部聲譽，而不是你在報紙上的位置，這才是真正重要的。

And for a long time, open AI didn't really have any titles except for always a CEO title, right?

而在很長一段時間裡，開放式人工智能並沒有真正意義上的頭銜，只有一個首席執行官的頭銜，不是嗎？

But didn't really have a lot of titles within the organization itself.

但在組織內部並沒有什麼頭銜。

People always knew who the great researchers were.

人們總是知道誰是偉大的研究者。

Once you have the scaling laws and certainly how AI research is being done now, there's sort of this shift where basically scale is all you need for increasingly more and more AI domains.

一旦你掌握了擴展規律，當然還有現在人工智能研究的方式，就會出現這樣一種轉變，即對於越來越多的人工智能領域來說，基本上規模就是你所需要的一切。

It's sort of potentially coming true in image diffusion models or in earlier to bring it back to what you're starting out with.

它有可能在影像擴散模型或更早的模型中實現，使其回到你開始時的狀態。

There's some sense that similar principles to scaling laws actually do apply in the right domains in robotics.

從某種意義上說，類似於縮放定律的原則確實適用於機器人技術的正確領域。

Is that sort of one of the things that you're seeing or how would you respond to that?

這是你看到的情況之一嗎？你會如何應對？

I think if you look at AI progress, you see scaling laws all over the place.

我認為，如果你看看人工智能的進步，就會發現到處都是縮放規律。

And so the interesting question is, well, if scaling laws exist and they're commonplace, what does that mean?

是以，有趣的問題是，好吧，如果縮放法存在並且司空見慣，這意味著什麼？

What does that mean for you if you're a company, if you're a researcher, if you're trying to make things better?

如果你是一家公司，如果你是一名研究人員，如果你想讓事情變得更好，這對你意味著什麼？

Why didn't we take advantage of scaling laws earlier in these other domains?

為什麼我們沒有更早地在這些其他領域利用縮放定律呢？

Well, I think we were really trying to.

嗯，我想我們真的在努力。

Usually in order to, the first step is actually getting to a scaling law.

通常情況下，第一步要做的就是找到一個縮放規律。

To take an example that's not LLMs.

舉個不是法學碩士的例子。

If you think about Dolly, which was how do you take text and make an image out of it?

想想《多莉》吧，你是如何把文字變成影像的？

I think Aditya Ramesh who built that model spent 18 months, maybe two years just getting to the first version that clearly worked.

我認為，建立該模型的 Aditya Ramesh 花了 18 個月、甚至兩年的時間，才完成了第一個明確可行的版本。

So I remember he'd be working on this and Ilya would come and show me, he'd be like, Aditya's been working on this for a year.

所以我記得他在做這個，伊利亞會過來給我看，他會說，阿迪提亞已經做了一年了。

He's trying to make a pink panda.

他想做一隻粉紅色的熊貓。

That's skating on ice because it's something that's clearly not in the training set.

這是在冰上滑行，因為這顯然不在訓練集中。

And here's an image and you can see it's like pink up there and white down there.

這是一張圖片，你可以看到上面是粉紅色，下面是白色。

It's really beginning to work.

它真的開始起作用了。

And I would look at that.

我會看看這個。

I'd be like, really?

我會想，真的嗎？

I mean, maybe, maybe, I don't know.

我是說，也許，也許，我不知道。

But just getting to that point where it's sort of plausibly begins to work is a huge difficult problem.

但是，要達到這樣的程度，讓它似是而非地開始工作，卻是一個巨大的難題。

And it's completely separate from using scaling laws.

這與使用縮放定律完全不同。

Now, once you get it to work, that's when scaling laws come into play.

現在，一旦你讓它工作起來，這就是縮放定律發揮作用的時候了。

And with scaling laws, you have two hard things that you can do.

有了縮放法，你可以做兩件難事。

One of them is just the pure scale itself.

其中之一就是純量表本身。

Scaling is not easy.

擴大規模並非易事。

It is in fact, probably the practical problem in any sort of model building.

事實上，這可能是任何一種模型製作中的實際問題。

And it's a systems problem.

這是一個系統問題。

It's a data problem.

這是一個數據問題。

It's an algorithmic problem.

這是一個算法問題。

Even if you're just trying to scale the same architecture.

即使你只是試圖擴展相同的架構。

The second thing you can do is you can try to change the slope of the scaling law or just bump it up a little bit.

你可以做的第二件事是嘗試改變縮放定律的斜率，或者把它調高一點。

And that is searching for better architectures, searching for better optimization algorithms, all of the algorithmic improvements that you can do.

這就是尋找更好的架構，尋找更好的優化算法，以及所有你能做到的算法改進。

And if you put all of those together, that is what explains the very fast progress that we're seeing in AI today.

如果把所有這些放在一起，這就是我們今天看到的人工智能飛速發展的原因所在。

I guess that is one of the bigger debates that's ongoing certainly out there in the community.

我想這也是目前社會上爭論的焦點之一。

Are the scaling laws going to continue to hold or are we hitting some sort of bottlenecks?

縮放規律是否會繼續保持，或者我們是否會遇到某種瓶頸？

I don't know how much you can talk about it, but what's your view at this point on maybe LLM scaling, but certainly other domains too?

我不知道你能談多少，但目前你對法律碩士的規模，當然也包括其他領域有什麼看法？

It is definitely the case that there is a data wall and that if you take the same techniques that we were using to scale LLMs, at some point you're going to run into that.

數據牆肯定是存在的，如果採用與我們擴展 LLM 相同的技術，在某些時候就會遇到數據牆。

The thing that's been really exciting, of course, is going from the LLM scaling of pre-training where you're just bringing bigger and bigger corpuses and trying to predict the next token and shifting gears and using techniques like reasoning, which OpenAI has shipped and it's 01 and 03 models and Gemini has now also shipped in Gemini Flash thinking.

當然，真正令人興奮的事情是從 LLM 的預訓練擴展而來，在這種預訓練中，你只需引入越來越大的語料庫，並嘗試預測下一個標記，然後換擋並使用推理等技術，OpenAI 已經推出了 01 和 03 模型，Gemini 現在也在 Gemini Flash thinking 中推出了這種技術。

If you think about Moore's law, with Moore's law, Moore's law is one big exponential curve, but it's actually the sum of a bunch of little S-curves.

如果你想想摩爾定律，摩爾定律是一條大的指數曲線，但它實際上是一堆小 S 曲線的總和。

And you start off with Dennard scaling and at some point that breaks.

一開始，丹納德會進行縮放，但到了一定程度，縮放就會中斷。

But if you think about how NVIDIA has gone, Moore's law has continued.

但是，如果你仔細想想英偉達的發展歷程，就會發現摩爾定律一直在延續。

It's just come through a different mechanism.

它只是通過不同的機制產生的。

So you solve some bottleneck, but then you S-curve that particular solution.

是以，你解決了一些瓶頸問題，但隨後你又對該特定解決方案進行了 S 曲線分析。

But there are other places where there are other bottlenecks.

但還有一些地方存在其他瓶頸。

And then you have a new bottleneck and you have to go attack that.

然後你又遇到了新的瓶頸，你必須去解決它。

And so, I think in the big picture, we're reaching that bottleneck for pre-training and data.

是以，我認為從全局來看，我們已經達到了預培訓和數據的瓶頸。

Are we exactly there?

我們真的到了嗎？

It's a little hard to say.

這有點難說。

But now we have this new mechanism with reasoning and test-time compute.

但現在，我們有了這種新的推理和測試時間計算機制。

I think if you go back and you think about what it took for AI, for building AGI, for, I would say the last five years, people have thought that, people at the big frontier labs have felt that, you know, step one was pre-training and that the remaining gap to have something that could scale all the way to AGI was reasoning.

我認為，如果你回過頭來思考人工智能、AGI 的構建過程，在過去的五年裡，人們認為，大型前沿實驗室的人們認為，第一步是預訓練，剩下的差距就是推理，這樣才能一直擴展到 AGI。

Some ability to take the same pre-trained model and have the ability to give it more time to think or more compute of various kinds and get a better answer at the other end.

有能力使用相同的預訓練模型，給它更多時間思考或進行更多的各種計算，並在另一端得到更好的答案。

And now that that has been cracked, at this point, I think we actually have a very clear path to just focus on scaling.

現在，我們已經破解了這一難題，在這一點上，我認為我們實際上已經有了一條非常清晰的道路，只需專注於擴大規模。

You know, we were, you know, talking about that, you know, the zero to one part that's not about scaling.