- There have been a lot of news about ChatGPT lately

- 最近有很多關於ChatGPT的消息

like people using ChatGPT to write essays,

像人們使用ChatGPT來寫論文。

ChatGPT hitting a hundred million users,

ChatGPT的用戶數達到了一億人。

Google launching Bard to compete against ChatGPT

谷歌推出巴德，與ChatGPT競爭

and Microsoft integrating ChatGPT

和微軟整合ChatGPT

into all their products, and also the viral sensation

在他們所有的產品中，也有病毒性的轟動。

of CatGPT where it can answer all of your queries,

的CatGPT，它可以回答你所有的疑問。

but as a cat, meow, meow, meow, meow, meow, meow.

但作為一隻貓，喵、喵、喵、喵、喵、喵、喵。

ChatGPT, if you don't know already, it's a chat bot

ChatGPT，如果你還不知道，它是一個哈拉機器人。

by OpenAI where you can ask it many things.

由OpenAI提供，在那裡你可以問它很多事情。

For example, explaining complex topics

例如，解釋複雜的主題

like explain why I'm a disappointment to my parents

就像解釋為什麼我對我的父母很失望一樣

or ask it more technical questions like,

或問它更多的技術問題，如。

how do I inherit more money than my brother from my parents?

我怎樣才能從父母那裡繼承比我弟弟更多的錢？

A lot of people are using it to write essays, draft emails,

很多人都在用它來寫論文，起草電子郵件。

and even write code.

甚至是寫代碼。

So I tried it myself, of course, as a YouTuber obviously,

所以我自己也試了一下，當然是作為一個優酷網友。

my first question to it was, who is Joma Tech?

我對它的第一個問題是，Joma Tech是誰？

And it answered...

而它的回答是...

Are you fucking--

你他媽的...

You know, ChatGPT has a lot of limitations,

你知道，ChatGPT有很多限制。

like here we ask it to name colors

就像在這裡，我們要求它命名顏色

that don't have the letter E in them,

不含字母E的。

and this is what they gave us.

而這就是他們給我們的東西。

Orang, yllow, red, that's clearly wrong.

橙色、黃色、紅色，這顯然是錯誤的。

In all seriousness,

認真地說。

this is to demonstrate how ChatGPT works.

這是為了演示ChatGPT是如何工作的。

It's a pre-trained large language model,

這是一個預先訓練好的大型語言模型。

meaning it was trained on text data

意味著它是在文本數據上訓練的

from the internet until the end of 2021.

在2021年年底之前，從互聯網上獲取信息。

So it won't know anything

所以它不會知道任何事情

about things that happened recently.

關於最近發生的事情。

It doesn't have access to the internet.

它沒有接入互聯網。

It'll only predict the answer based

它只會根據以下情況預測答案

on what it has consumed already,

在它已經消耗的東西上。

and the way it answers your question is

而它回答你的問題的方式是

by predicting each word that comes next.

通過預測接下來的每一個單詞。

For example, if you ask GPT who Bard is,

例如，如果你問GPT，巴德是誰？

it's not going to know.

它是不會知道的。

You might ask Joma, didn't your channel launch in 2017

你可能會問Joma，你的頻道不是在2017年推出嗎？

and ChatGPT was trained on internet data until 2021,

而ChatGPT在2021年之前都是根據互聯網數據進行訓練。

yet it doesn't know who you are?

但它卻不知道你是誰？

Yeah, so there's actually a technical reason

是的，所以實際上有一個技術原因

and fuck you.

和他媽的你。

Recently ChatGPT hit a hundred million users.

最近ChatGPT的用戶數達到了一億。

It launched November 30th, 2022,

它於2022年11月30日啟動。

and this article came out February 3rd, 2023.

而這篇文章是在2023年2月3日發表的。

So it took two months to hit a hundred million users.

是以，它花了兩個月的時間就達到了一億用戶。

Who are these users and what are they doing with ChatGPT?

這些用戶是誰，他們在用ChatGPT做什麼？

Well, it's pretty obvious, they're cheating with it.

嗯，這很明顯，他們在用它作弊。

Everybody's cheating such that

每個人都在作弊，這樣

some school districts have banned access to ChatGPT.

一些學區已經禁止訪問ChatGPT。

If they can write essays, then they can pass exams.

如果他們能寫論文，那麼他們就能通過考試。

ChatGPT was able to pass exams from law school,

ChatGPT能夠通過法律學校的考試。

business school, and medical school.

商學院和醫學院。

Three prestigious industries.

三個著名的行業。

Now, this is why I went into coding

現在，這就是我進入編碼領域的原因

because I always thought that law school,

因為我一直認為，法律學校。

business school, and medical school,

商學院和醫學院。

it was too much about memorization

太多關於記憶的東西了

and you're bound to get replaced,

而你一定會被替換。

it just wasn't intellectual enough, you know?

它只是不夠聰明，你知道嗎？

All right, well,

好了，好了。

I guess engineering is getting replaced, too.

我想工程也在被取代。

ChatGPT passes Google coding interview,

ChatGPT通過了谷歌的編碼面試。

which is known to be hard, but I guess not.

眾所周知，這是很難的，但我想不是。

But note that it is for a L3 engineer,

但請注意，這是針對L3級工程師的。

which means it's a entry level, for those not in tech,

這意味著它是一個入門級，對那些不從事技術工作的人來說。

there's no L2 and L1, it starts at L3,

沒有L2和L1，它從L3開始。

but this does raise questions about ChatGPT's ability

但這確實讓人對ChatGPT的能力產生懷疑。

to change engineering jobs behind it,

以改變它背後的工程工作。

and we're already seeing the change

而且我們已經看到了這種變化

as Amazon employees are already using ChatGPT

因為亞馬遜員工已經在使用ChatGPT

for coding even though that immediately after,

為編碼，即使是緊接著。

they told them to stop, warning them not

他們叫他們停下來，警告他們不要

to share confidential information with ChatGPT.

與ChatGPT分享機密信息。

What's happening is they're feeding ChatGPT

現在的情況是他們在給ChatGPT提供食物

internal documents, which are confidential,

內部文件，這些文件是保密的。

but OpenAI stores all that data.

但OpenAI存儲了所有這些數據。

You know, it reminds me of when I used to intern

你知道，這讓我想起了我以前實習的時候

at Microsoft and they didn't let us use Google

在微軟，他們不允許我們使用谷歌。

for searches because they think that they might spy on us.

因為他們認為他們可能會監視我們，所以要進行搜查。

I was like, relax, I'm an intern.

我當時說，放鬆，我是個實習生。

I'm not working on anything important.

我沒有在做任何重要的工作。

In fact, I actually wasn't working at all.

事實上，我實際上根本就沒有工作。

You know, I was playing Overwatch all day,

你知道，我整天都在玩《守望先鋒》。

but yeah, anyways, they forced us to use Bing for searches.

但是，無論如何，他們強迫我們使用Bing進行搜索。

One thing that's being underreported

有一件事沒有被充分報道

in mainstream media is the success of GitHub Copilot.

主流媒體的報道是GitHub Copilot的成功。

It's probably the most useful

這可能是最有用的

and most well executed AI product currently out there.

和目前執行得最好的人工智能產品。

Have I used it?

我用過嗎？

No, I haven't coded in forever.

不，我已經很久沒有編碼了。

Now, here's how it works.

現在，事情是這樣的。

The moment you write your code,

在你寫代碼的那一刻。

it's like auto complete on steroids, like this example,

它就像類固醇的自動完成，就像這個例子。

it helps you write the whole drawScatterplot function

它可以幫助你編寫整個drawScatterplot函數

and it knows how to use a D3 library correctly.

而且它知道如何正確使用D3庫。

Another example here, you can write a comment

這裡還有一個例子，你可以寫一個評論

explaining what you want your function to do

解釋你希望你的函數做什麼

and it'll write the code for you.

它就會為你寫代碼。

Sometimes even the name

有時甚至連名字

of the function will give it enough information

的函數會給它足夠的資訊

to write the rest of the code for you.

來為你寫其餘的代碼。

It's very powerful

它是非常強大的

because it's able to take your whole code base as context

因為它能夠把你的整個代碼庫作為上下文。

and with that, make more accurate predictions that way.

並以此為基礎，做出更準確的預測。

For example, if you're building a trading bot

例如，如果你正在建立一個交易機器人

and you write the function get_tech_stock_prices,

而你寫了函數get_tech_stock_prices。

it'll suggest, hey, I know you're going

它將暗示，嘿，我知道你要去

through a rough time,

通過一個艱難的時期。

but building a trading bot is not going

但建立一個交易機器人並不是要

to fix your insecurities and maybe you should just accept

來解決你的不安全感，也許你應該接受

that you'll be a disappointment for the rest of your life.

你會在你的餘生中成為一個令人失望的人。

Okay.

好的。

How did all of this happen?

這一切是如何發生的？

Why is AI so good suddenly?

為什麼人工智能突然變得這麼好？

The answer is the transformer model

答案是變壓器模型

which caused a paradigm shift

這引起了範式的轉變

on how we build large language models, LLM.

關於我們如何建立大型語言模型，LLM。

By the way, this diagram means nothing to me.

順便說一句，這張圖對我來說毫無意義。

It makes me look smart, so that's why I put it on there.

它使我看起來很聰明，所以這就是我把它放在上面的原因。

Before transformers,

在變壓器之前。

the best natural language processing system used RNN,

最好的自然語言處理系統使用了RNN。

and then it used LSTM,

然後，它使用了LSTM。

but then Google Brain published a paper

但後來谷歌大腦發表了一篇論文

in 2017 called "Attention is All You Need"

在2017年，名為 "關注是你所需要的一切"

which is also my life's motto because I'm a narcissist.

這也是我的人生格言，因為我是一個自戀者。

The paper proposes a simple neural network model

本文提出了一個簡單的神經網絡模型

they call transformer, which is based

他們稱之為變壓器，它是基於

on the self attention mechanism

關於自我注意機制

which I don't fully understand, so I'll pretend

我並不完全理解，所以我就假裝

like I don't have time to explain it

就像我沒有時間去解釋它一樣

but I also know that it allows for more parallelization

但我也知道，它可以實現更多的並行化

which means you can throw more hardware,

這意味著你可以扔更多的硬件。

more GPUs to make your training go faster

更多的GPU，使你的訓練更快進行

and that's when things got crazy.

就在這時，事情變得瘋狂起來。

They kept adding more data and also added more parameters

他們不斷添加更多的數據，也添加更多的參數

and the model just got better.

而且該模型剛剛變得更好。

So what did we do?

那麼我們做了什麼？

We made bigger models with more parameters

我們做了更大的模型，有更多的參數

and shoved it a shit ton of data.

並把一噸的數據塞給它。

Sorry, I'm trying my best here to make the model bigger.

對不起，我在這裡盡力使模型變大。

All right, fuck it.

好吧，去他媽的。

Anyway, that gave us ready

總之，這讓我們準備好了

to use pre-trained transformer models like Google's Bert,

來使用預先訓練好的轉化器模型，如谷歌的Bert。

and OpenAI's GPT, generative pre-trained transformers.

和OpenAI的GPT，生成性預訓練的轉化器。

They crawled the whole web to get text data

他們抓取了整個網絡來獲得文本數據

from Wikipedia and Reddit.

來自維基百科和Reddit。

This graph shows you how many parameters each model has.

該圖顯示了每個模型有多少個參數。

So as you can see, we've been increasing the number

是以，正如你所看到的，我們一直在增加

of parameters exponentially.

的參數呈指數增長。

So OpenAI kept improving their GPT model

所以OpenAI不斷改進他們的GPT模型

like how Goku kept becoming stronger each time

就像悟空每次都會變得更強

he reached a new Super Saiyan form.

他達到了一個新的超級賽亞人形態。

While editing this,

在編輯這個的時候。

I realized how unhelpful the "Dragon Ball" analogy was.

我意識到 "龍珠 "的比喻是多麼的無助。

So I want to try again.

所以我想再試試。

To recap, transformer was the model architecture,

簡而言之，變壓器是模型架構。

a type of neural network.

一種類型的神經網絡。

Other types of models would be like RNN and LSTM.

其他類型的模型將像RNN和LSTM。

Compared to RNN, transformers don't need

與RNN相比，變壓器不需要

to process words one by one,

來逐一處理單詞。

so it's way more efficient at training with lots of data.

所以它在大量數據的訓練中更有效率。

OpenAI used the transformer model and pre-trained it

OpenAI使用了轉化器模型並對其進行了預訓練

by feeding it a bunch of data from the internet

通過從互聯網上輸入一堆數據來實現。

and they called that pre-trained model GPT-1.

他們把這個預訓練的模型稱為GPT-1。

Back then, NLP models would be trained from scratch

那時，NLP模型會從頭開始訓練

for a specific task like translation or summarization.

為一項特定的任務，如翻譯或總結。

Both transformer, we get to pre-train the model first

兩個變壓器，我們都要先對模型進行預訓練

then fine tune it for a specific task.

然後針對具體任務進行微調。

Then for GPT-2, they did the same thing, but more

然後對於GPT-2，他們做了同樣的事情，但更多的是

and with a bigger model, hence with 1.5 billion parameters,

並有一個更大的模型，是以有15億個參數。

and then with GPT-3,

然後用GPT-3。

they went crazy and gave it 175 billion parameters.

他們瘋了，給了它1750億的參數。

However, just like raising a kid,

然而，就像養育一個孩子一樣。

just shoving it with a bunch

只是用一群人推著它

of information unsupervised might not be the best way

無監督的資訊可能不是最好的方式。

to raise a kid.

來撫養一個孩子。

She might know a lot of things,

她可能知道很多事情。

but she hasn't learned proper values from her parents.

但她還沒有從父母那裡學到正確的價值觀。

So that's why we have to fine tune it, tell it what's right,

所以這就是為什麼我們必須對它進行微調，告訴它什麼是正確的。

and what's wrong, how not to be racist and clean up its act.

以及什麼是錯的，如何不成為種族主義者並清理其行為。

That's GPT-3.5, a more fine-tuned version of GPT-3

這是GPT-3.5，是GPT-3的一個更微調的版本。

with guardrails that can be released to the public.

有護欄，可以向公眾發佈。

Now you have a decently well-behaved kid,

現在你有一個體面的乖巧的孩子。

but you now want to show her off, so you dress it up nicely,

但你現在想向她炫耀，所以你把它打扮得很好。

get her ready for her first job, AKA more fine tuning

讓她為她的第一份工作做好準備，又稱更多的微調。

with some supervised training

有一些監督的培訓

so it behaves properly as a chat bot.

這樣它就能正確地作為一個哈拉機器人行事。

That way it's well packaged and is ready to ship

這樣一來，它就被包裝得很好，可以隨時發貨。

to the world with a web UI.

通過網絡用戶界面向世界展示。

Okay, back to the original shitty "Dragon Ball" explanation.

好吧，回到最初的低劣的 "龍珠 "解釋上。

So you can think of Goku's hair,

所以你可以想到悟空的頭髮。

like the number of parameters, 175 billion parameters,

像參數的數量，1750億個參數。

which is why you can see Goku has more hair now.

這就是為什麼你可以看到悟空現在有更多的頭髮。

Goku hair isn't much longer,

悟空的頭髮並沒有多長。

but it's just styled a little bit differently.

但它只是在風格上有一點不同。

100 trillion parameters.

100萬億個參數。

So technically GPT-3 was already amazing

所以技術上來說，GPT-3已經很了不起了

but OpenAI was able to package it neatly with ChatGPT

但OpenAI能夠將其與ChatGPT整齊地打包。

which made it user friendly, so it became a viral sensation.

這使得它對用戶友好，所以它成為一種病毒式的轟動。

So yeah, packaging is important.

所以，是的，包裝很重要。

It caused everyone to really pay attention to this.

這引起了大家對此事的真正關注。

So how did people react to the viral growth of ChatGPT?

那麼，人們對ChatGPT的病毒式增長有什麼反應？

People were mind blown and said, Google is done

人們心花怒放，說，谷歌已經完成了

because ChatGPT is going to replace search engines.

因為ChatGPT將取代搜索引擎。

No, it can't.

不，它不能。

Until it can search for porn,

直到它能搜索到色情。

it cannot replace search engines.

它不能取代搜索引擎。

Oh, wait, why search for porn

哦，等等，為什麼要搜索色情

when you could generate it?

當你能產生它的時候？

(upbeat music)

(歡快的音樂)

Anyway, even losing a bit of search volume

無論如何，即使失去一點搜索量

to ChatGPT would be a big deal for Google

到ChatGPT將是谷歌的一個大事件。

since 80% of their revenue comes from ads

因為他們80%的收入來自於廣告

and most of it comes from search.

而其中大部分來自於搜索。

People were telling Google to release something similar.

人們告訴谷歌要發佈類似的東西。

Google was like, bruh, we have LaMDA,

谷歌就像，哥們兒，我們有LaMDA。

which is basically ChatGPT, but releasing it would be risky

這基本上是ChatGPT，但發佈它將是有風險的。

as they had much more reputational risk at stake

因為他們面臨的聲譽風險要大得多

and has to move more conservatively than a startup would.

並且必須比初創公司更保守地行動。

That's foreshadowing by the way.

順便說一句，這就是預示。

Microsoft is chilling.

微軟令人心寒。

They positioned themselves really well

他們把自己定位得非常好

by investing $1 billion in OpenAI early on in 2019.

通過在2019年初向OpenAI投資10億美元。

That allowed OpenAI to leverage Microsoft's Azure

這使得OpenAI能夠利用微軟的Azure

for its compute power to train and run their models

因為它的計算能力可以訓練和運行他們的模型

and Microsoft gets to integrate OpenAI's tech

和微軟得到整合OpenAI的技術

into their products.

融入他們的產品。

So if OpenAI succeeds,

是以，如果OpenAI成功了。

Microsoft succeeds and remember GitHub Copilot?

微軟成功了，還記得GitHub Copilot嗎？

Well, GitHub is owned by Microsoft, so that's a huge win.

好吧，GitHub是由微軟擁有的，所以這是一個巨大的勝利。

Meanwhile, Google is panicking

與此同時，谷歌正在恐慌中

and issued a code red,

併發出了紅色代碼。

calling in the OG founders Page and Brin.

召集OG創始人佩奇和布林。

Actually I have no idea who's who, so...

事實上，我不知道誰是誰，所以......。

Anyways, but they called them to strategize

不管怎麼說，但他們叫他們去制定戰略

on how to approach this.

關於如何處理這個問題。

Microsoft is fueling the momentum, especially

微軟正在為這一勢頭推波助瀾，特別是

with ChatGPT growing so fast

隨著ChatGPT的快速增長