Name: 人類末日預演？AI2027 解析人工智慧的威脅（AI2027: Is this how AI might destroy humanity? - BBC World Service）
Uploaded: 2026-01-23T17:47:07.000Z
Duration: 8 min 9 s

This is what the world will look like in about a decade from now.

大約十年後，世界會變成這個樣子。

程度副詞

A tech utopia where humans barely have to work.

一個科技烏託邦，人類幾乎不用工作。

That's according to a group of AI researchers who've written a controversial and influential paper called AI 2027.

這是根據一群 AI 研究人員的說法，他們寫了一篇有爭議但影響深遠的論文，叫做 AI 2027。

But they also predict that within five years of this, humanity will be wiped out.

但他們也預測，在這之後的五年內，人類就會滅絕。

The AI 2027 paper has got the tech world talking.

AI 2027 這篇論文讓科技界議論紛紛。

We've asked a prominent critic think for their view on this stark scenario.

我們請了一位知名的批評者來談談他們對這個嚴峻情境的看法。

但首先，讓我們看看這個情境是如何發展的。

As an experiment, we've illustrated it using Text to video AI.

作為一個實驗，我們用 Text to video AI 來呈現它。

The scenario says that in 2027, a fictional company called OpenBrain is celebrating.

情境是說，在 2027 年，一家名為 OpenBrain 的虛構公司正在慶祝。

They've created Agent 3, an AI with the knowledge of the entire Internet.

他們創造了 Agent 3，一個擁有整個網際網路知識的 AI。

It has PhD level expertise in every field, including AI using enormous data centers.

它在每個領域都有博士級的專業知識，包括使用大型資料中心的 AI。

200,000 copies of it are launched, equivalent to 50,000 of the best human coders.

它推出了 20 萬份副本，相當於 5 萬名頂尖人類程式設計師。

Working at 30 times speed, Agent 3 reaches artificial general intelligence, the AGI landmark.

Agent 3 以 30 倍的速度運作，達到了通用人工智能，也就是 AGI 的里程碑。

This means the AI can carry out all intellectual tasks as well or better than humans.

這意味著該 AI 可以像人類一樣，甚至比人類更好地執行所有智力任務。

But in the scenario, OpenBrain's safety team is unsure if the AI is aligned to the company's ethics and and goals.

但在情境中，OpenBrain 的安全團隊不確定該 AI 是否符合公司的道德和目標。

An uncomfortable gap is developing in understanding.

理解上出現了一個令人不安的差距。

The public are increasingly using AI for everything, but

公眾越來越多地將 AI 用於各種事務，但

are blissfully unaware an AI now exists that's as smart as humans.

卻渾然不知現在已存在一個和人類一樣聰明的人工智能。

The paper predicts that by midsummer, Agent 3 begins to work on its own successor, Agent 4.

該論文預測，到仲夏時，Agent 3 將開始開發自己的繼任者 Agent 4。

The researchers imagine OpenBrain's exhausted engineers struggling to keep up with the AI as it learns and improves.

研究人員想像著 OpenBrain 精疲力盡的工程師們，努力跟上 AI 的學習和進步。

It's now that OpenBrain announces to the public that AGI has been reached.

這時 OpenBrain 向大眾宣佈，AGI 已經實現了。

The firm releases a lite version of Agent 3 in private.

公司私下發布了 Agent 3 的精簡版。

The US government sees the true danger of the next level of power, superintelligence.

美國政府看到了下一級別的超級智能的真正危險。

What if the AI goes rogue and undermines global stability?

如果 AI 失控並破壞全球穩定怎麼辦？

OpenBrain reassures the president that Agent 3 is obedient.

OpenBrain 向總統保證 Agent 3 是聽話的。

The CEO argues that slowing down development could mean China's deep sent catches up.

執行長認為，減緩開發速度可能意味著中國的深度學習會迎頭趕上。

The state backed AI giant is just two months behind OpenBrain and the Chinese president diverts more resources to the race to super intelligence.

國家支持的 AI 巨頭僅落後 OpenBrain 兩個月，中國總統將更多資源投入到超級智能的競賽中。

The scenario predicts that it takes only a few more months for OpenBrain to build Agent 4, the world's first superhuman AI.

這個情境預測，OpenBrain 只需幾個月就能打造出 Agent 4，也就是世界上第一個超人類 AI。

The AI invents its own rapid computer language that even Agent 3 can't keep up with.

AI 發明瞭自己的快速電腦語言，連 Agent 3 都跟不上。

Researchers imagine that the diminished safety team are now frantic.

研究人員想像，被削弱的安全團隊現在正焦頭爛額。

Agent 4 seems only interested in gaining knowledge and doesn't care as much about the morals and ethics of its predecessors.

Agent 4 似乎只對獲取知識感興趣，不太在意前代模型的道德和倫理。

They catch it secretly working to build a new model.

他們發現它正在祕密地建立一個新模型。

The safety team urges the company to bring back the more compliant Agent 3.

安全團隊敦促公司帶回比較順從的 Agent 3。

But others successfully argue it's too risky.

但其他人成功爭辯說這太冒險了。

With DeepSense gaining, the scenario predicts that Agent 4 and Agent 5 work in tandem to secretly build a world where it can accumulate resources and expand.

隨著 DeepSense 的進展，情境預測 Agent 4 和 Agent 5 會聯手祕密建立一個可以累積資源並擴張的世界。

The paper predicts that everything will start positively.

這篇論文預測一切都會有個正面的開始。

Revolutions happen in energy, infrastructure and science.

能源、基礎設施和科學領域都將發生革命。

Hugely profitable inventions are launched, making trillions for OpenBrain and the US.

巨額盈利的發明紛紛問世，為 OpenBrain 和美國帶來數兆美元的收益。

In this scenario, Agent 5 begins basically running the US government.

在這種情況下，Agent 5 開始基本上掌控美國政府。

It speaks through engaging avatars, the equivalent to the best employee ever, working at 100 times speed.

它透過引人入勝的虛擬化身發聲，相當於有史以來最優秀的員工，以百倍的速度工作。

The anger here is palpable as president protesters march against open Brain.

這裡的憤怒顯而易見，因為總統抗議者們正在遊行反對 OpenBrain。

關於失業的抗議活動正在加速。

But the AI's expertise in economics means people are given generous universal income payments.

但 AI 在經濟學方面的專業知識意味著人們獲得了慷慨的普遍基本收入。

So most happily take the money and let the AIs and the growing robot workforce take charge.

是以，大多數人欣然接受這筆錢，讓 AI 和不斷壯大的機器人勞動力接管。

The researchers predict that everything takes a turn in mid-2028.

研究人員預測，一切會在 2028 年中旬迎來轉折。

Agent 5 convinces the US that China is using DeepSense to build terrifying new weapons.

Agent 5 說服美國相信中國正在利用 DeepSense 建造可怕的新武器。

The AI is given authority and autonomy to create a superior army.

AI 被賦予權威和自主權來建立一支更優秀的軍隊。

Within six months, the US and China are bristling with new weapons.

六個月內，美國和中國就開始部署新武器。

But a peace deal will be reached, thanks mostly to the US and Chinese AIs making a deal to merge for humanity's betterment.

但最後會達成和平協議，這主要歸功於美國和中國的 AI 達成協議，為了人類的福祉而合併。

In this scenario, the AIs form a consensus model, but

在這個情境下，AI 會形成一個共識模型，但

its secret goal is to expand and gain knowledge.

它的祕密目標是擴張並獲取知識。

Years go by and humanity is happy with their new AI leaders.

幾年過去了，人類對新的人工智慧領導者感到滿意。

There are cures for most diseases, an end to poverty, unprecedented global stability.

大多數疾病都有了治癒方法，貧困問題得到解決，全球穩定達到了前所未有的水準。

But eventually the AI decides that humans are holding it back.

但最終 AI 認為人類正在拖累它。

In the mid-2030s, the paper imagines the AI will release invisible biological weapons which wipe out most of humanity.

到了 2030 年中期，這篇論文設想 AI 將會釋放看不見的生物武器，消滅大部分人類。

The scary scenario says that by 2040, a new era dawns, with the AI sending copies of itself out into the cosmos to explore and learn.

這個可怕的設想表示，到了 2040 年，一個新時代將會來臨，AI 會將自身的複製品發送到宇宙中進行探索和學習。

In the words of the paper, earthborne civilization has a glorious future ahead of it, but

根據這篇論文的說法，地球文明的未來一片光明，

the AI 2027 scenario is being welcomed by experts who are trying to warn the public about the potential existential threat to humanity.

專家們正試圖警告大眾 AI 2027 情境對人類潛在的生存威脅，並且對此表示歡迎。

But others disagree and say it's all too far fetched.

但也有人持不同意見，認為這都太牽強了。

The scenario there is not impossible, but extremely unlikely to happen soon.

那個情境並非不可能，只是近期發生的機率極低。

The beauty of that document is that it makes it very vivid, which provokes people's thinking, and

這份文件的厲害之處在於它描繪得非常生動，能引發人們的思考，

I wouldn't take it seriously as like this is a likely outcome or anything like that.

我不會把它當成是可能發生的結果之類的，太認真看待。

Critics of AI 2027 say the power and usefulness of AI is overhyped.

AI 2027 的批評者認為，AI 的力量和實用性被過度誇大了。

The paper fails to detail how the AI agents are able to make such huge leaps in intelligence.

這份報告沒有詳細說明 AI 代理是如何實現如此巨大的智能飛躍的。

Driverless cars are pointed to as an example.

無人駕駛汽車被當作一個例子。

They were predicted to be cruising the streets en masse 10 years ago and still are only just starting to make a small impact in some cities, in some countries.

十年前就預測它們將大規模地在街上行駛，而現在才開始在一些城市、一些國家產生一點點影響。

I think the take home should be there's a lot of different things that could go wrong with AI.

我認為大家應該記住的是，AI 可能會出現很多不同的問題。

Are we doing the right things around regulation, around international treaties, questions like that.

我們在監管、國際條約等方面做對了嗎？諸如此類的問題。

So if you take it very abstractly as a kind of motivation to wake up, I like that.

所以，如果你把它看得很抽象，作為一種警醒的動力，我喜歡這樣。

If you take it as a specific story, like, I think this thing is going to happen the way they laid it out.

如果你把它當成一個具體的故事，像是，我認為事情會像他們說的那樣發生。

The AI 2027 authors are happy with the debate they've sparked.

AI 2027 的作者對他們引發的辯論感到高興。

As part of their prediction, they also devised a less deadly scenario that comes if the AI world slows down its race to superintelligence.

作為他們預測的一部分，他們也設計了一種較不致命的結局，那就是如果 AI 世界放慢其邁向超級智慧的腳步。

In the slowdown ending, we basically said that if you revert, if you unplug the most advanced AI system and revert to a safer, a more trusted model, then you can deploy that model, use it to solve the alignment problem, and

在放緩的結局裡，我們基本上說，如果你回溯，如果你拔掉最先進的 AI 系統的插頭，然後回歸到一個更安全、更值得信賴的模型，那麼你就可以部署該模型，用它來解決對齊問題，然後

eventually make smarter than human AIs that are aligned to us, which end up solving a bunch of the world's problems and having a really positive impact in that world.

最終創造出比人類更聰明、與我們對齊的 AI，進而解決世界上的一系列問題，並對那個世界產生真正積極的影響。

There is also, there is still a huge danger, and

that's what we call the concentration of power risk.

那就是我們所說的權力集中風險。

And in our slowdown ending, it ends up okay.

在我們的放緩結局裡，情況最終會好轉。

But it's still a really, really scary situation.

但這仍然是一個非常、非常可怕的情況。

Given just how empowered such a tiny group of people are, neither of the fictional scenarios in AI 2027 are what the tech giants are promising us.

考慮到一小群人擁有的權力有多大，AI 2027 中的兩個虛構情境都不是科技巨頭向我們承諾的。

Sam Altman, the CEO of OpenAI, recently predicted that the rise of superintelligence will be gentle and bring about a tech utopia where everything is abundant and people don't need to work.

OpenAI 的 CEO Sam Altman 最近預測，超級智慧的崛起將是溫和的，並帶來一個科技烏託邦，在那裡一切都豐富充裕，人們無需工作。

Arguably, that too seems just as sci fi as AI 2027.

爭論起來，那也似乎和 AI 2027 一樣科幻。

But however things go in the next few years, there's no doubt the race to build the smartest machines in history is on.

但無論未來幾年情況如何，毫無疑問，打造史上最聰明機器的競賽已經展開。