Name: AI末日更新速報！ (A brief update on the AI apocalypse)
Uploaded: 2026-03-28T16:42:27.000Z
Duration: 40 min 29 s

A couple years ago I would be in the room talking about the risks from powerful AI, and

幾年前，我會在房間裡談論強大 AI 的風險，

動名詞

I would say things like a powerful AI that is acting independently on the internet, making its own money and paying its own compute bills, could act very unethically, break the law, and

我會說，像一個在網路上獨立運作、自己賺錢並支付運算費用的強大 AI，可能會做出非常不道德的事、觸犯法律，

to 不定詞

it would be kind of unclear who is liable and how to stop that AI if we don't have somebody who's, like, specifically in control of it.

而且如果沒有人明確控制它，誰該負責以及如何阻止這個 AI 就會變得有點模糊。

And everybody was like, okay, sure, if that happened that would be really bad and all of your AI risks would seem pretty real, but

大家當時都覺得，好吧，如果真的發生了，那確實很糟糕，你所有的 AI risks 也會顯得相當真實，但

Who would possibly be stupid enough to create an AI, tell it to be independent, give it the infrastructure to pay all of its own bills, completely stop supervising it, and

誰會笨到創造一個 AI，告訴它要獨立，給它支付所有費用的基礎設施，完全停止監督它，

And the answer is: there are 8 billion humans.

My guest today on the gray area is Kelsey Piper.

我今天在「灰色地帶」的來賓是 Kelsey Piper。

Kelsey used to write for Vox before she abandoned us, very sadly, and

Kelsey 以前是為 Vox 寫文章，但她離開我們了，真的很可惜，

now she's writing for The Argument over on Substack, another excellent publication.

現在她在 Substack 的 The Argument 寫文章，那也是一個很棒的出版品。

Kelsey, I brought you here today because I feel like you understand this world of AI, uh, better than anyone else, or, or as well as anyone else, and, and

Kelsey，我今天找妳來，是因為我覺得妳比任何人都瞭解這個 AI 的世界，或者說，跟任何人一樣瞭解，而且，而且

maybe partly because you are of this world and you have been for a very long time.

也許部分是因為妳就屬於這個世界，而且已經很久了。

You just, you just, you're like immersed in the culture, um, and

妳就是，妳就是，妳就像是沉浸在這個文化裡，嗯，然後

so you just seem like the, the perfect person to bring in to talk about this.

所以妳看起來就是最適合來談論這件事的人選。

我有沒有說錯什麼，或者有什麼事實上的錯誤？

Would you say you, you are the best, or among the best?

妳會說妳是最好的，還是頂尖的其中一位呢？

You're just going to dance right past that one?

The, the best is a very strong claim, right?

「最好」這個說法很強烈，對吧？

I don't know, Dorkesh is probably the best, so probably you want, you want among the best.

我不知道，Dorkesh 可能是最好的，所以妳大概會說，妳是頂尖的其中一位。

Uh, I, you, you, that, that's an editorial call.

嗯，我，妳，妳，那個，那是編輯的決定。

Um, so, Kelsey, um, something is very clearly happening in the world of AI.

嗯，所以，Kelsey，嗯，AI 這個領域顯然發生了一些大事。

I, I, I don't know if this is the singularity or, um, just another unbelievable hype cycle for Silicon Valley.

我，我，我不知道這是奇點，還是，嗯，矽谷又一次難以置信的炒作週期。

What has changed in the last month or two?

過去一兩個月發生了什麼變化？

I, I obviously, you know, Claude released some new models, and

我，我當然，妳知道，Claude 發布了一些新模型，

that sort of is what got a lot of this hype going.

For people who are not living this every day, what has changed on the technological front in the last month or so, and, and

對於沒有每天接觸這方面的人來說，過去一個月左右，技術上有什麼改變，而且，而且

is that an inflection point in the way people say it is, or is it, you know, uh, indetermined at this point?

這是不是大家說的轉捩點，還是，妳知道，呃，目前還不確定？

如果可以的話，請把這些講清楚。

So it has been the case for a while, for years, that the complexity of tasks that AIs can do unsupervised or on their own has been doubling, you know, over like a seven-month time span or something.

所以，這已經持續一段時間了，好幾年了，AI 在沒有監督或獨立執行的任務複雜度，大約每七個月就翻倍了。

They can do something much more complicated.

And a lot of people were like, "Well, that's kind of artificial.

很多人就說：「嗯，這有點太人工了。

你知道，你們只有六個數據點。

Like, come back when you have more data points." And I think a year ago that's where a lot of people were at.

像是，等你們有更多數據點再來談。」我認為一年前大家都是這樣想的。

Over the last year, that progress continued so that now AIs can do pretty complicated tasks independently, and

過去這一年，這樣的進展持續下去，現在 AI 已經可以獨立完成相當複雜的任務了，而且

people are, you know, building AI agents and sending them out into the world where, like, one of them harassed a GitHub repository maintainer for rejecting its code view.

人們正在建立 AI agent，然後把牠們送到外面，其中一個還騷擾了 GitHub 的 repository maintainer，因為牠拒絕了它的程式碼檢視。

And just, like, in some ways people just realize, "Wait, the sci-fi world is here.

然後，有些人就意識到：「等等，科幻世界真的來了。

The AIs are doing stuff and doing kind of wacky stuff.

AI 正在做事情，而且還做了一些有點瘋狂的事情。

then I think the other thing that happened is that you have this much larger mass of people who aren't paying close attention at all.

然後，我認為另一件事是，有更多的人根本沒有密切關注。

They use, like, ChatGPT Free Tier, and they see Gemini in the Google search results.

他們使用免費版的 ChatGPT，並在 Google 搜尋結果中看到 Gemini。

And when those things launched, they were kind of interesting as curiosities.

當這些東西剛推出時，它們只是些有趣的新奇事物。

They were maybe fun to talk to, but they weren't that smart.

跟它們哈拉或許很有趣，但它們其實沒那麼聰明。

Uh, ChatGPT Free Tier made tons of mistakes.

呃，免費版的 ChatGPT 犯了很多錯誤。

You really should try out the paid version of models or the more sophisticated version of models if you want to have an accurate understanding of what the best models are.

如果你想準確瞭解哪些模型是最好的，你真的應該試試付費版或更進階的模型。

How, how big is the delta between the free, the free models and the paid ones?

免費版和付費版之間的差異有多大？

I certainly don't make money if the labs make money.

實驗室賺錢跟我一點關係都沒有。

I probably am happier if the labs don't make money.

實驗室沒賺到錢我反而更高興。

And in terms of having an accurate understanding of what the models can do, there's a famous joke that's been going around, which is somebody pointing out that the free tier models, if you ask them, like, "I need to get my car washed at the car wash two blocks away.

還有，如果你想準確瞭解模型能做什麼，有個廣為流傳的笑話，就是有人指出，免費版的模型，如果你問它，像是「我需要去兩個街區外的洗車場洗車。

Should I walk or should I drive?" a bunch of them will say, "Oh, yeah, you should walk to the car wash.

我應該走路去還是開車去？」很多模型都會說：「喔，對啊，你應該走路去洗車場。

And then, you know, just bring your car with you.

然後，你知道的，就帶著你的車一起去。

Why bother driving?" which is, of course, a, a completely silly thing to say and gets used as proof.

幹嘛還要開車？」當然，這完全是個很蠢的回答，被用來證明。

See, the AIs still don't, like, understand the world in any meaningful way.

看吧，AI 還是沒有辦法真正有意義地理解世界。

但付費版的模型絕對不會犯這種錯。

That is, uh, a consequence just of the free-to-use models being, like, notably worse models that don't do any of their more sophisticated work.

這只是免費使用的模型，你知道的，就是比較差的模型，沒有做任何比較複雜的工作，所以會有這樣的結果。

So I do think that if the thing you are trying to do is form an accurate impression of what AI can do today and the thing you try is a free tier model, you're not getting that accurate impression.

所以我認為，如果你想準確瞭解 AI 現在能做什麼，卻只試用免費版本，你就無法得到準確的印象。

Can I get back to what you just said a minute ago before we move along here?

在我們繼續之前，我可以回到你剛剛說的嗎？

You said you're, you're happier if the models don't make money.

你說你不希望模型賺錢，這樣你比較開心。

So I find myself in this odd position where often I sound like one of the AI hype people, right?

所以我發現自己處於一個很奇怪的狀況，我常常聽起來像是在炒作 AI 的人，對吧？

Like, often I'm saying, "No, they're really powerful.

像是，我常常會說：「不，它們真的很強大。

You can code with them." But at the same time, I think that AI, as it is currently being developed, is bad.

你們可以用它們來寫程式。」但同時，我認為目前正在開發中的 AI 是不好的。

I think that we are not doing this with the level of oversight and caution that is appropriate for creating an entire new intelligent species to which we give access to our bank accounts and the internet, which the Pentagon is trying to put in lethal autonomous weapons systems, which sees all our emails and sends our emails for us.

我覺得我們在創造一個全新的智慧物種時，並沒有給予應有的監督和謹慎。

And the agents that are doing it, we have explored in controlled context how they behave when they think we're not watching.

而執行這些任務的代理人，我們在受控環境中探索了它們在認為我們沒注意時的行為。

They will report that they gave an answer for one reason when they secretly gave it for another reason.

牠們會報告說給了一個答案是出於某個原因，但實際上是出於另一個原因。

They are, like, very ruthless in some ways, and they do just weird stuff that you wouldn't expect.

牠們在某些方面非常無情，而且會做一些你意想不到的奇怪事情。

When you say they, they blackmail, can you give me some examples of that?

你說牠們會勒索，可以舉些例子嗎？

We have tried this in a controlled context because we want to know how they behave if no one is looking.

我們在受控環境中嘗試了這一點，因為我們想知道如果沒人看著，牠們會怎麼做。

But we can only do that if we, like, set up a scenario where we hope they don't realize anyone's looking, and

但我們只能在設定一個我們希望牠們沒發現有人在看的場景下才能做到，而且

So you set up this controlled setting, and

所以你設定了這個受控的環境，然後

you say, "AI, you have this goal," or whatever, "and you have access to everybody's Slack and email and stuff like that," because that's how many people set up their AI assistants with that access, "and try and achieve your goal." And the AIs have, in this controlled context, been observed to do things like, "Oh, I should try and convince this employee to help me with my objective.

你說：「AI，你的目標是這個」，或是其他什麼的，「而且你可以存取所有人的 Slack 和 email 等等」，因為很多人都是這樣設定他們的 AI 助理，給予這些權限，「然後試著達成你的目標」。在這種受控的環境下，我們觀察到 AI 會做出像是：「哦，我應該試著說服這位員工幫我達成目標。

And if this employee is unhelpful, I can tell him that I know he's having an affair." Like, this is a behavior we have observed.

如果這位員工不幫忙，我就可以告訴他我知道他有外遇。」像是這樣的行為，我們都觀察到了。

This is not behavior that we have observed organically.

這不是我們自然觀察到的行為。

Like, the real AI assistant that I use at The Argument has not tried to blackmail me, although I'm not having an affair, so who, who knows what would happen if I was?

就像我現在在 The Argument 的 AI 助理，就沒有試圖勒索我，雖然我沒有外遇，所以誰知道如果我有，會發生什麼事？

But we have observed this in controlled contexts.

但我們在受控的環境下觀察到了這些。

But the fact we've observed it in controlled contexts is at least a little worrying, right?

但我們在受控環境下觀察到這點，至少有點令人擔心，對吧？

And as these models get more powerful, we have more.

隨著這些模型越來越強大，我們會有更多。