Placeholder Image

字幕列表 影片播放

  • I'm going to talk about my research

    今天要談論的是我做的研究

  • on the long term future of artificial intelligence.

    有關人工智慧的長遠未來

  • In particular, I want to tell you

    尤其,我想跟你們介紹

  • about a very important phenomenon called "Intelligence Explosion."

    一個極其重要的現象,稱為「智能大爆炸」

  • There are two reasons that I work on intelligence explosion

    我鑽研智能大爆炸的原因有兩個

  • and that I think it's worth sharing.

    也是我認為值得分享的原因

  • The first is that it's a phenomenon of immense theoretical interest

    第一,此現象背後的理論引起很多興趣

  • for those who want to understand intelligence on a fundamental level.

    對於那些想要從基本面向理解人工智能的人

  • The second reason is practical.

    第二個原因是實際應用

  • It has to do with the effects that intelligence explosion could have.

    這與智能大爆炸可能造成的效果有關

  • Depending on the conditions

    依照不同

  • under which an intelligence explosion could arise

    智能大爆炸可能產生的情形

  • and on the dynamics that it exhibits

    以及呈現出的改變

  • it could mean that AI changes very rapidly

    足以顯示人工智能的迅速變化

  • from a safe technology, relatively easy to handle,

    從安全無害的科技,相當容易掌控

  • to a volatile technology that is difficult to handle safely.

    一直到多變化的科技,就變得相當難以安全掌控

  • In order to navigate this hazard,

    為了釐清風險

  • we need to understand intelligence explosion.

    我們必須了解智能大爆炸

  • Intelligence explosion is a theoretical phenomenon.

    智能大爆炸是一個理論性現象

  • In that sense, it's a bit

    意義上來說,這就像

  • like a hypothetical particle in particle physics.

    粒子物理學中,假設會存在的粒子

  • There are arguments that explain why it should exist,

    有很多論點來解釋它的真實性

  • but we have not been able to experimentally confirm it yet.

    但我們仍無法透過實驗驗證

  • Nevertheless, the thought experiment

    然而,此思想實驗

  • that explains what intelligence explosion would look like

    用來解釋智能大爆炸會是什麼情況

  • is relatively simple.

    其實相當簡單扼要

  • And it goes like this.

    實驗是這樣進行的

  • Suppose we had a machine

    假設我們有一台機器

  • that was much more capable than today's computers.

    比現代電腦的性能更卓越

  • This machine, given a task,

    給予這台機器一項任務

  • could form hypotheses from observations,

    能夠藉由觀察建立假說

  • use those hypotheses to make plans, execute the plans,

    使用這些假說擬訂計畫,執行計畫

  • and observe the outcomes relative to the task,

    再觀察工作結果

  • and do it all efficiently within a reasonable amount of time.

    在合理的時間內有效率地完成所有工作

  • This kind of machine could be given science and engineering tasks

    這種機器能夠執行科學以及工程工作

  • to do on its own, autonomously.

    而且全自動化

  • And this is the key step in the thought experiment:

    這在思想實驗中是極關鍵的一步

  • this machine could even be tasked with performing AI research,

    這台機器甚至能夠執行人工智能研究

  • designing faster and better machines.

    設計出速度更快、性能更好的機器

  • Let's say our machine goes to work, and after a while,

    我們命令機器開始工作,過不久

  • produces blueprints for a second generation of AI,

    機器就能設計出第二代人工智能的藍圖

  • that's more efficient, more capable, and more general than the first.

    性能、效率、規格都比前一代更加出色

  • The second generation can be tasked once again

    第二代能夠再次被指派工作

  • with designing improved machines,

    要設計更好的機器

  • leading to a third generation, a fourth, a fifth, and so on.

    創造第三代、第四、第五代,延續下去

  • An outside observer would see

    旁觀者角度來看

  • a very large and very rapid increase in the abilities of these machines,

    這些機器的能力獲得卓越成長,十分迅速

  • and it's this large and rapid increase

    這種卓越又迅速的成長

  • that we call Intelligence Explosion.

    我們稱為「智能大爆炸」

  • Now if it's the case

    假設

  • that in order to undergo an intelligence explosion

    為了要產生智能大爆炸

  • many new pieces of hardware need to be build,

    必須打造出很多新的硬體零件

  • or new manufacturing technologies,

    或是打造新的製造科技

  • then an explosion will be more slow

    如此一來,智能大爆炸就會更慢

  • - although still quite fast by historical standards.

    即便以歷史來看,其成長仍然相當快

  • However, looking at the history of algorithmic improvement

    然而,觀察演算法過去的演變

  • it turns out that just as much improvement

    事實證明,這種硬體上的進步

  • tends to come from new software as from new hardware.

    透過軟體,也同樣達到類似的進步

  • This is true in areas like physics simulation, game playing,

    某些領域也是如此,例如:物理性質模仿、遊戲

  • image recognition, and many parts of machine learning.

    圖像識別,以及許多有關機器學習的部分

  • What this means is that our outside observer may not see physical changes

    這意味著旁觀者可能沒有看到

  • in the machines that are undergoing an intelligence explosion.

    發生智能大爆炸時,機器外觀上的變化

  • They may just see a series of programs

    他們可能僅看見許多電腦程式

  • writing successively more capable programs.

    不斷寫出更多性能更佳的電腦程式

  • It stands to reason that this process could give rise to programs

    按理說,這個進步過程會導致電腦程式

  • that are much more capable at any number of intellectual tasks than any human is.

    更能夠勝任腦力工作,甚至比人類更加卓越

  • Just as we now build machines that are much stronger, faster, and more precise

    如同我們現在創造了更加強勁、快速、精準的機器

  • at all kinds of physical tasks,

    能勝任所有勞力工作

  • it's certainly possible to build machines

    那肯定也有可能建造一種機器

  • that are more efficient at intellectual tasks.

    能夠在腦力工作更有效率

  • The human brain is not at the upper end of computational efficiency.

    人腦尚未達到運算效率的極致

  • And it goes further than this.

    然而不只如此

  • There is no particular reason

    目前尚沒有理由

  • to define our scale by the abilities of a single human or a single brain.

    以一個人或一個腦的能力去限制能力範圍

  • The largest thermonuclear bombs release more energy

    世界最大的氫彈釋放出更多能量

  • in less than a second

    在短短一秒之內

  • than the human population of Earth does in a day.

    比起全地球的人類一天之內的釋放總量更多

  • It's not out of the question to think

    有件事是肯定的

  • that machines designed to perform intellectual tasks

    用來執行腦力工作的機器

  • and then honed over many generations of improvement

    再經歷數個世代的成長

  • could similarly outperfom

    最終足以勝過

  • the productive thinking of the human race.

    人類的思考能力

  • This is the theoretical phenomenon called Intelligence Explosion.

    這就是所謂「智能大爆炸」的假設理論

  • We don't have a good theory of intelligence explosion yet,

    目前仍未發展出關於智能大爆炸的理論

  • but there is reason to think that it could happen at software speed

    但有理由認為這件事可能一觸即發

  • and could reach a level of capability

    軟體能夠快速成長至相當程度的性能

  • that's far greater than any human or group of humans

    遠比任一個人類或一群人類

  • at any number of intellectual tasks.

    在執行腦力工作時更加卓越

  • The first time I encountered this argument,

    我首次聽到這理論時

  • I more or less ignored it.

    其實並不重視

  • Looking back it seems crazy for me, someone who takes AI seriously,

    回頭看才發覺自己非常瘋狂,我是如此嚴肅的看待人工智慧

  • to walk away from intelligence explosion.

    一開始還十分不屑智能大爆炸

  • And I'll give you two reasons for that.

    我分享兩個原因

  • The first reason is a theorist's reason.

    第一是身為一位理論家

  • A theorist should be interested in the large-scale features of their field

    理論家應該著重於自身專業領域的大概念

  • in the contours of their phenomena of choice as determined by

    專注於他們想觀察的現象的原理

  • the fundamental forces, or interactions, or building blocks of their subject.

    基於他們學科中的基礎力量、相互作用、或基礎學問

  • As someone who aspires to be a good theorist of intelligence,

    身為一個渴求成為人工智慧的優秀理論家的人

  • I can't, in good faith, ignore intelligence explosion

    我無法不把智能大爆炸

  • as a major feature

    當作一個重要概念

  • of many simple straightforward theories of intelligence.

    由許多人工智能的簡單基礎理論而來

  • What intelligence explosion means

    智能大爆炸的意思是

  • is that intelligence improvement is not uniform.

    人工智慧的成長沒有規律

  • There is a threshold below which improvements tend to peter out,

    這種成長有一道門檻,令幅度逐漸下滑

  • but above that threshold,

    然而在此門檻之上

  • intelligence grows like compound interest increasing more and more.

    人工智慧會像複利一樣呈倍數成長

  • This threshold would have to emerge from

    這道門檻必定是從

  • any successful theory of intelligence.

    人工智慧的成功理論中誕生

  • The way phase transitions emerge from thermodynamics,

    就像相位變化是從熱力學發展出來的一樣

  • intelligence would effectively have a boiling point.

    類似地,人工智慧也會有這個突破口

  • Seeing this way,

    基於這種方式

  • exploring intelligence explosion is exactly the kind of thing

    探究智能大爆炸就是一項

  • a theorist wants to do, especially in a field like AI,

    理論家想研究的事情,尤其是人工智慧的領域

  • where we are trying to move from our current state

    我們領域目前極力想脫離現在的研究的方向

  • ,partial theories, pseudotheories, arguments, and thought experiments,

    ,片面理論、偽理論、學說、思想實驗,

  • toward a fully-fledged predictive theory of intelligence.

    朝向一個發育健全的前瞻性智能理論

  • This is the intelligence explosion.

    這就是智能大爆炸

  • In its most basic form,

    最根本的道理就是

  • it relies on a simple premise

    仰賴一個簡單的基礎

  • that AI research is not so different from other intellectual tasks

    人工智慧研究與其他腦力工作其實相差不遠

  • but can be performed by machines.

    它們都能夠由機器執行

  • We don't have a good understanding yet,

    我們尚未瞭如指掌

  • but there's reason to think that it can happen at software speed

    但我們有理由認為它能在軟體運行速度內達到

  • and reach levels of capability

    達到相當程度的性能

  • far exceeding any human or group of humans.

    超越任一個人類或一群人類

  • The second reason which I alluded to at the start of the talk

    第二個原因要回到我開頭提到的

  • is that intelligence explosion could change AI very suddenly

    智能大爆炸能夠迅速改變人工智慧

  • from being a benign technology to being a volatile technology

    從無危險性的科技轉變為極為不穩的科技

  • that requires significant thought into safety

    這得從安全層面好好思考

  • before use or even development.

    在尚未研發或是使用這項科技之前

  • Today's AI, by contrast, is not volatile.

    現代的人工智慧是相對無危險性的

  • I don't mean that AI systems can't cause harm.

    我不是指這些人工智慧系統毫無危害

  • Weaponization of AI is ongoing, and accidental harms can arise

    人工智慧武器化正持續進行,而意外的危害可能

  • from unanticipated systemic effects or from faulty assumptions.

    從意料之外的系統結果或是錯誤判定而發生

  • But on the whole, these sorts of harms should be manageable.

    總而言之,這類的危害應該都能夠控制

  • Today's AI is not so different from today's other technologies.

    現今的人工智慧與現在的其他科技並沒有太大差異

  • Intelligence explosion, however highlights an important fact:

    然而智能大爆炸卻強調了一個重要的事實:

  • AI will become more general, more capable, and more efficient

    人工智慧將變得更普及、性能更卓越、效率更好

  • perhaps very quickly

    或許很快就發生了

  • and could become more so than any human or group of humans.

    而且能夠超越所有人類

  • This kind of AI will require

    這種人工智慧將需要

  • a radically different approach to be used safely.

    一個截然不同的方式來使用,以確保安全

  • And small incidents could plausibly escalate to cause large amounts of harm.

    即使是一件小事或許最終也能引發成大危害

  • To understand how AI could be hazardous,

    要理解人工智慧造成危害的方式就是

  • let's consider an analogy to microorganisms.

    思考一下微生物的例子

  • There are two traits

    有兩點特徵

  • that make microorganisms more difficult to handle safely than a simple toxin.

    與一個單純的毒素相比,這使得微生物難以安全掌控

  • Microorganisms are goal-oriented,

    微生物以目標為導向

  • and they are, what I'm going to call, chain reactive.

    還有他們其實有所謂的連鎖反應

  • Goal-oriented means

    目標為導向指的是那些

  • that a microorganisms behaviors

    微生物的行為

  • tend to push towards some certain result.

    會朝向某些特定結果努力

  • In their case that's more copies of themselves.

    在微生物的例子來說,目標就是要多複製自己

  • Chain reactive means

    連鎖反應指的是

  • that we don't expect a group of microorganisms to stay put.

    我們並未預期這群微生物按兵不動

  • We expect their zone of influence to grow,

    我們預期它們的影響範圍擴增

  • and we expect their population to spread.

    也預期他們繁殖

  • Hazards can arise, because a microorganisms

    會產生危害,因為微生物

  • values don't often align with human goals and values.

    的價值與人類目標及價值時常並不相符

  • I don't have particular use

    我並沒有特別

  • for an infinite number of clones of this guy.

    想要不斷複製這個微生物

  • Chain reactivity can make this problem worse.

    連鎖反應會讓問題惡化

  • Since, small releases of a microorganism can balloon

    因為少量的微生物能夠

  • into large population spending pandemics.

    大量繁殖

  • Very advanced AI, such as could arise from intelligence explosion,

    極先進的人工智慧,而且是從智能大爆炸崛起的

  • could be quite similar in some ways to a microorganism.

    將會和微生物的某些地方有極為相似之處

  • Most AI systems are task-oriented.

    大部分的人工智慧系統是以任務為導向

  • They are designed by humans to complete a task.

    他們由人類設計以完成任務

  • Capable AIs will use many different kinds of actions

    有能力的人工智慧將會使用不同的行為

  • and many types of plans to accomplish their tasks.

    以及多樣的計畫完成任務

  • And flexible AIs will be able to learn to thrive,

    有彈性的人工智慧將能夠學習並且成長

  • that is to make accurate predictions and effective plans

    最後就能夠做出精準預測以及有效規劃

  • in a wide variaty of environments.

    能夠適應任何環境

  • Since AIs will act to accomplish their tasks as well as possible,

    因為人工智慧會極力完成所交代的任務

  • they will also be chain reactive.

    所以會產生連鎖反應

  • They'll have use for more resources, they'll want to improve themselves,

    它們將使用更多資源,極欲自我成長

  • to spread to other computer systems, to make backup copies of themselves

    帶動其他電腦程式,一直備份自己

  • in order to make sure that their task gets done.

    目的就是確保能夠完成任務

  • Because of their task orientation and chain reactivity,

    由於它們的任務導向以及連鎖反應

  • sharing an environment with this kind of AI would be hazardous.

    這種情況下的人工智慧會有危害

  • They may use some of the things we care about,

    它們會利用我們在乎的事物

  • our raw materials, and our stuff to accomplish their ends.

    我們的原物料以及其他原料都會被拿去完成目標

  • And there is no task that has yet been devised

    這種情況下,還沒有任務被設計為

  • that is compatible with human safety under these circumstances.

    能兼顧人類安全

  • This hazard has made worse by intelligence explosion,

    這種危害在智能大爆炸之後會更加惡化

  • in which very volatile AI could arise quickly from benign AI.

    良好的人工智慧將快速變成有害、容易失控的人工智慧

  • Instead of a gradual learning period,

    並非是漸進學習

  • in which we come to terms with the power of very efficient AI,

    那種我們習慣於高效人工智慧的能力

  • we could be thrust suddenly into a world

    我們會突然進入到一個世界

  • where AI is much more powerful than it is today.

    到時候人工智慧將會史無前例的強大

  • This scenario is not inevitable,

    這種情況並非無可避免

  • it's mostly dependent upon

    主要取決於

  • some research group, or company, or government

    某些研究團隊、企業或政府

  • walking into intelligence explosion blindly.

    對於智能大爆炸的盲目、不了解

  • If we can understand intelligence explosion,

    若我們能夠理解智能大爆炸

  • and if we have sufficient will and self-control as a society,

    還能夠團結一心,有足夠的信念以及自我控制的能力

  • then we should be able to avoid an AI outbreak.

    我們就能夠阻止人工智慧爆發

  • There is still the problem of chain reactivity though.

    即便仍存在連鎖反應的問題

  • It would only take one group to release AI into the world

    只要一組團隊釋出了人工智慧

  • even if nearly all groups are careful.

    即使其它組都非常小心翼翼

  • One group walking into intelligence explosion accidently or on purpose

    只要有一組不小心,或故意地造成了智能大爆炸

  • without taking proper precautions,

    在毫無準備時

  • could release an AI that will self-improve

    將會釋放出能夠自我成長的人工智慧

  • and cause immense amounts of harm to everyone else.

    對於所有人造成龐大的危機

  • I'd like to close with four questions.

    我要用四個問題總結

  • These are questions that I'd like to see answered

    我樂見這些問題能夠被回答

  • because they'll tell us more about the theory of artificial intelligence

    因為他們告訴我們更多有關人工智慧的理論

  • and that theory is what will lead us understand intelligence explosion

    正是這理論使我們了解智能大爆炸

  • well enough to mitigate the risks that it poses.

    讓我們足以降低它所產生的風險

  • Some of these questions are being actively pursued

    當中有些問題正被

  • by researchers at my home institution,

    我母校的研究學者著手處理

  • The Future of Humanity Institute at Oxford,

    牛津大學的人類未來機構

  • and by others, like The Machine Intelligence Research Institute.

    以及其他機構,例如機器智能研究機構

  • My first question is,

    第一個問題是:

  • "Can we get a precise predictive theory of intelligence explosion?"

    「我們能否得出智能大爆炸的精確預測理論?」

  • What happens when AI starts to do AI research?

    如果人工智慧著手研究人工智慧會發生什麼事?

  • In particular, I'd like to know

    我特別想了解

  • how fast software can improve its intellectual capabilities.

    程式能多快地提升自己的智力

  • Many of the most volatile scenarios we've examined include

    我們所研究的危險情境包括了

  • a rapid self-contained take off,

    一個程式能自給自足,快速的開始

  • such as could only happen under a software improvement circumstance.

    例如,只能發生在程式自我提升的情況當中

  • If there is some key resource that limits software improvement

    如果有一些關鍵資源能限制程式進步

  • or if it's the case that such improvement isn't possible

    或是程式提升的情況並不太可能

  • below a certain threshold of capability,

    發生在特定的門檻以下

  • these would be very useful facts from a safety standpoint.

    從安全的角度來看,這會是非常有用的事實

  • Question two:

    第二個問題:

  • what are our options, political or technological,

    我們有哪些選擇,不管是政治上或是技術面

  • for dealing with the potential harms

    能處理從一個超級高效的人工智慧

  • from super efficient artificial intelligences?

    所帶來的潛在危害

  • One option, of course, is to not build them in the first place.

    第一個選擇,當然就是一開始就不要創造它們

  • But this would require exceedingly good cooperation

    但這將會需要極端地合作

  • between many governments, commercial entities, and even research groups.

    在許多政府、商業團體甚至研究團隊

  • That cooperation and that level of understanding isn't easy to come by.

    這種合作性以及共識程度不容易獲得

  • It would also depend, to some extent, on an answer to question one

    這某種程度上也取決於問題一的答案

  • so that we know how to prevent intelligence explosion.

    讓我們知道如何避免智能大爆炸

  • Another option would be to make sure

    另一個選項是確保

  • that everyone knows how to devise safe tasks.

    每個人都了解如何設計安全任務