Name: 鮑勃-麥克格魯：人工智能代理和通向 AGI 的道路 (Bob McGrew: AI Agents And The Path To AGI)
Uploaded: 2025-01-31T21:38:45.000Z
Duration: 31 min 2 s

to 不定詞

The deadline to apply for the first YC Spring Batch is February 11th.

第一批青年志願者春季班的申請截止日期是 2 月 11 日。

未來簡單式

If you're accepted, you'll receive $500,000 in investment, plus access to the best startup community in the world.

如果您被錄取，您將獲得 500,000 美元的投資，還可以進入世界上最好的初創企業社區。

祈使句型

So apply now and come build the future with us.

現在就申請吧，和我們一起創造未來。

If you ask people what AGI was, they would say it's a model that you can actually interact with.

如果你問人們什麼是 AGI，他們會說這是一個可以真正與之互動的模型。

And if you said, okay, well, what happens when you get all those capabilities?

如果你說，好吧，那麼當你獲得所有這些能力時會發生什麼呢？

Say, well, everybody's out of a job and game over for humanity.

說，好吧，每個人都失業了，人類的遊戲結束了。

I think in the big picture, we're reaching that bottleneck for pre-training and data.

我認為，從大局來看，我們正在達到預培訓和數據的瓶頸。

But now we have this new mechanism with reasoning and test time compute.

但現在，我們有了這種新的機制，可以進行推理和測試時間計算。

What we're going to see out of reasoning is that it's really going to unlock the possibility of agents to do actions on your behalf, which has sort of always been possible, but it's just never been quite good enough.

我們將從推理中看到的是，它將真正釋放代理人代表你採取行動的可能性。

嘿，夥計們，今天有好戲看了。

Bob McGrew, formerly chief research officer at OpenAI.

Bob McGrew，OpenAI 前首席研究官。

You were a part of building a lot of the research team.

早期在 OpenAI 工作是什麼感覺？

The really interesting thing about OpenAI is that I did not originally intend to go to a research lab.

OpenAI 真正有趣的地方在於，我原本並不打算去研究實驗室。

When I left Palantir, I wanted to start a company.

離開 Palantir 時，我想創辦一家公司。

I had a thesis that robotics would be the first real business that was built out of deep learning.

我曾有過這樣一個論斷，即機器人技術將是第一個通過深度學習建立起來的真正業務。

And I talked my way into a friend's nonprofit.

我還說服了一個朋友加入了非營利組織。

I never had a badge, but I would go in, he'd open the door for me.

我沒有徽章，但我進去後，他會給我開門。

And I learned deep learning by teaching a robot how to play checkers from vision.

我還通過教機器人如何從視覺出發下跳棋來學習深度學習。

And in the process of doing this, I learned a lot about robotics.

在這個過程中，我學到了很多關於機器人的知識。

And I learned that robotics was definitely not the right startup to start in 2015 or 2016.

我還了解到，機器人技術絕對不是 2015 年或 2016 年創業的正確選擇。

I ended up going to OpenAI basically because it was a place full of very smart people and it had big ambitions.

我最終去了 OpenAI，基本上是因為這裡有很多非常聰明的人，而且有很大的抱負。

It was a place where I could really learn.

在這裡，我可以真正學到東西。

I had all this management experience from Palantir, but it was just a place for me to really become an expert in deep learning.

我在 Palantir 積累了豐富的管理經驗，但那只是讓我真正成為深度學習專家的地方。

And from there, figure out what it could actually be used and applied for.

然後，再找出它的實際用途和適用範圍。

What were some of the earliest things that you remember working on?

在您的記憶中，最早的工作是什麼？

And how did that play into what everyone knows OpenAI to be now?

這又是如何影響大家對 OpenAI 的認識的？

When OpenAI started, the goal was always to build AGI.

OpenAI 創立之初，目標始終是構建 AGI。

But the theory early on was that we would build AGI by doing a lot of research and writing a lot of papers.

但早期的理論是，我們將通過大量研究和撰寫大量論文來構建 AGI。

我們知道這是一個錯誤的理論。

I think for a lot of the early people who were startup people, Sam, Greg, myself, it felt painful and a little academic.

我認為，對於很多早期的創業者來說，比如山姆、格雷格和我自己，這感覺很痛苦，而且有點學術化。

But at the same time, it was what we could do at the time.

但同時，這也是我們當時所能做的。

And so some of the early projects, I worked on a robotics project where we took a robot hand, a humanoid robot hand, and we taught it to solve Rubik's Cube.

是以，在早期的一些項目中，我參與了一個機器人項目，我們用一隻機器手，一隻仿人機器手，教它解魔術方塊。

The idea in doing that was that if we could make the environments complicated enough, the artificial intelligence would be able to generalize out of the narrow domain it was taught and learn something more complicated, which was one of the ideas that later we see coming back with LLMs.

這樣做的想法是，如果我們能把環境變得足夠複雜，人工智能就能跳出狹隘的教學領域，學習更復雜的東西。

The other really early big project was solving Dota 2.

另一個真正的早期大項目是解決 Dota 2。

So there's a long history of solving games as a path towards building better AI, from Othello to Go.

是以，從黑白棋到圍棋，將解決遊戲問題作為構建更好人工智能的途徑由來已久。

And after beating Go, the next hardest set of games are actually video games.

而在戰勝圍棋之後，下一組最難的遊戲其實是電子遊戲。

They're not very classy, but they're a lot of fun.

And I can assure you that mathematically, they were harder.

我可以向你保證，從數學上講，它們更難。

And so DeepMind went after StarCraft, OpenAI went after Dota 2.

於是，DeepMind 開始追逐《星際爭霸》，OpenAI 開始追逐《Dota 2》。

And there was real insight that was generated there, which was that it really strengthened our belief that scale was the path to improving artificial intelligence.

我們從中獲得了真知灼見，這讓我們更加堅信，規模化是改進人工智能的必由之路。

That with Dota 2, the secret idea was that we could take huge amounts of experience and feed it into a neural network, and that the neural network would actually learn and generalize from that.

在《Dota 2》中，我們的祕密想法是，我們可以將大量的經驗輸入到神經網絡中，然後神經網絡就能從中學習和歸納。

And later, we actually went back and applied this to the robot hand, and that became the key idea for the robot hand.

後來，我們又把它應用到了機械手上，這也成為了機械手的關鍵創意。

And at the same time as these two big projects were going on, Alec Radford was experimenting with language.

在這兩個大項目進行的同時，亞歷克-拉德福德也在進行語言實驗。

And the core idea behind GPT-1 is that if you have a transformer, and you apply this super simple objective of guessing the next token, guessing the next word, that that would be enough signal that you could actually have something that would be able to generate coherent text.

GPT-1 背後的核心理念是，如果你有一個變壓器，並應用這個超級簡單的目標來猜測下一個標記，猜測下一個單詞，這將是足夠的信號，你實際上可以有一個能夠生成連貫文本的東西。

And in retrospect, it sounds sort of obvious, right?

現在回想起來，這聽起來很明顯，對嗎？

Like, you know, clearly this was going to work, but no one thought this would work at the time.

就像，你知道，這顯然是行得通的，但當時沒有人認為這行得通。

Alec really had to persevere for years in order to make this work, and that became GPT-1.

艾力克真的是堅持了好幾年才把這件事做成的，這就是後來的 GPT-1。

And then after GPT-1 seemed successful, we brought in the ideas from Dota and from the robot hand of training at larger and larger amounts of scale, and training on a really diverse set of data, and looking for generalization.

在 GPT-1 似乎取得成功後，我們引入了 Dota 和機器人手的理念，即以越來越大的規模進行訓練，在真正多樣化的數據集上進行訓練，並尋求泛化。

And together, that brings you to GPT-2, and GPT-3, and GPT-4.

合起來就是 GPT-2、GPT-3 和 GPT-4。

So one of the things that OpenAI really pioneered and sort of figured out was this concept of scale.

是以，OpenAI 真正開創並摸索出的一個概念就是 "規模"。

How is it that it was OpenAI that made the right decisions and sort of found large language models first?

為什麼是 OpenAI 做出了正確的決定，並首先找到了大型語言模型？

Early on, there were sort of a couple big projects, as I said, and then some room for exploratory research.

正如我所說，早期有幾個大項目，然後還有一些探索性研究的空間。

And at the very earliest days, the exploratory research was really about what the researcher wanted to do, but also it was about sort of the company's opinion.

在最開始的時候，探索性研究實際上是關於研究人員想要做什麼，同時也是關於公司的意見。

And in this, it was primarily formed by Ilya, with influence from a lot of people, but I think Ilya was really the guiding light here early on.

在這一點上，它主要是由伊利亞形成的，也受到了很多人的影響，但我認為伊利亞在早期確實是這裡的指路明燈。

Sometimes I think about the OpenAI culture, and I like to oppose it to sort of Google Brain and to DeepMind.

有時我在思考 OpenAI 文化，我喜歡把它與谷歌大腦和 DeepMind 對立起來。

And so early on, the DeepMind culture was a caricature.

是以，在早期，DeepMind 的文化就是一幅漫畫。

Demis had a big plan, and he wanted to hire a bunch of researchers so he could tell them to move forward with his plan.

戴米斯有一個大計劃，他想僱一批研究人員，這樣他就可以告訴他們推進他的計劃。

And Google Brain said, let's rebuild academia.

谷歌大腦說，讓我們重建學術界吧。

Let's like bring in all these super talented researchers.

讓我們把這些才華橫溢的研究人員都請來吧。

Let's just let them figure out what they want to do, give them lots of resources and hope that amazing products pop out.

讓他們自己想辦法吧，給他們提供大量資源，希望他們能創造出令人驚歎的產品。

And of course they did, but they didn't necessarily happen at Google.

當然，它們確實發生了，但不一定發生在谷歌。

And we took a different approach, which was really more like a startup, where there was no sort of big centralized plan, but at the same time, people didn't have, it wasn't just sort of, let's let a thousand flowers bloom.

我們採取了一種不同的方法，這種方法其實更像一家初創公司，沒有什麼集中式的大計劃，但與此同時，人們也沒有那種 "百花齊放 "的想法。

Instead, we had opinions about what needed to be done and things like, how do you show scale as a way of making your idea get better?

相反，我們對需要做的事情有自己的看法，比如，如何顯示規模才能讓你的想法變得更好？

And that opinion was set by the research leadership.

而這一意見是由研究領導層確定的。

Again, early on, people like Ilya, people like Dario, that was how we made sure that we didn't just sort of throw resources at everybody, but neither did we have just one set of ideas that were there.

同樣，在早期，像伊利亞這樣的人，像達里奧這樣的人，就是我們如何確保我們不只是把資源扔給每個人，但我們也不是隻有一套想法。

We found this sort of happy medium between the two.

我們在二者之間找到了一種平衡點。

I guess one of the critiques of maybe pure academia or some of the AI research labs, we don't have to name any of them, but we've heard stories about looking at the number of researchers on any given paper, there might be way more people on it.

我想，對純粹的學術界或一些人工智能研究實驗室的責備之一是，我們不必點名責備它們，但我們聽說過這樣的故事：從某篇論文的研究人員數量來看，可能有更多的人参與其中。

And if you really dig into some of the papers there, they look like maybe a little bit of this plus a little bit of that.

如果你仔細研究其中的一些論文，就會發現它們看起來就像是這個加一點那個。

And that sort of reflected the nature of that's what it took to get compute.

這也是其他人工智能實驗室的情況。

I mean, what was it about open AI where you were able to sort of avoid that?

我的意思是，在開放式人工智能中，你是如何避免這種情況的？

Well, I think the paper example is a really interesting example, because I think that's sort of both good and bad.

我認為紙張的例子非常有趣，因為我認為這既有好處也有壞處。

I am hugely positive on academics and researchers, but actually pretty negative on academia.

我對學者和研究人員的評價非常積極，但實際上對學術界的評價相當消極。

I think academia is good for this very narrow thing of small groups, trying out crazy ideas, but academia has a lot of incentives that prevent people from collaborating.

我認為，學術界對於小團體、嘗試瘋狂想法這種非常狹隘的事情來說是件好事，但學術界有很多阻礙人們合作的激勵機制。

And in particular in academia, there's this obsession with credit.

特別是在學術界，人們對學分有一種痴迷。

One of the things that's interesting about the way that papers have turned out in big labs is that early on we made the decision that we would try to be as Catholic as possible in putting everybody's name on it.

關於大型實驗室的論文成果，其中一件有趣的事情是，我們很早就做出決定，我們將盡可能地把每個人的名字都寫在論文上。

And one of the early robotics papers, we actually said cite as open AI because we didn't wanna get into a fight.

在早期的一篇機器人論文中，我們實際上說引用的是開放式人工智能，因為我們不想捲入爭論。

The first author is the one who gets cited and their name shows up every single time.

第一作者是被引用的人，他們的名字每次都會出現。

So we said, we're not gonna try to have this fight.

所以我們說，我們不打算打這場架了。

We're not gonna say who is the person who really did it.

我們只能說是開放式人工智能。

And I think that is actually a really important cultural piece, the ability to accept that people want credit, but to be able to channel it into, it's your internal reputation, not the position you have on the paper that really matters.

我認為，這實際上是一個非常重要的文化部分，即接受人們想要榮譽的能力，但能夠將其轉化為你的內部聲譽，而不是你在報紙上的位置，這才是真正重要的。

And for a long time, open AI didn't really have any titles except for always a CEO title, right?

而在很長一段時間裡，開放式人工智能並沒有真正意義上的頭銜，只有一個首席執行官的頭銜，不是嗎？

But didn't really have a lot of titles within the organization itself.

但在組織內部並沒有什麼頭銜。

People always knew who the great researchers were.

人們總是知道誰是偉大的研究者。

Once you have the scaling laws and certainly how AI research is being done now, there's sort of this shift where basically scale is all you need for increasingly more and more AI domains.

一旦你掌握了擴展規律，當然還有現在人工智能研究的方式，就會出現這樣一種轉變，即對於越來越多的人工智能領域來說，基本上規模就是你所需要的一切。