Is it okay if they stand over here?

如果他們站在這裡可以嗎？

- Yeah. - Um, actually.

- 是的 呃，實際上。

Christophe, if you can get even lower.

克里斯托弗，如果你能再低一點。

- Okay. - ( shutter clicks )

- 好的。 -（快門聲）。

This is Lee and this is Christophe.

這位是李，這位是克里斯托夫。

They're two of the hosts of this show.

他們是這個節目的兩位主持人。

But to a machine, they're not people.

但對機器來說，他們不是人。

This is just pixels. It's just data.

這只是像素而已。這只是數據而已。

A machine shouldn't have a reason to prefer

一臺機器不應該有理由去喜歡

one of these guys over the other.

這些人中的一個比另一個強。

And yet, as you'll see in a second, it does.

然而，正如你馬上就會看到的那樣，它確實如此。

It feels weird to call a machine racist,

說一臺機器是種族主義者，感覺很奇怪。

but I really can't explain-- I can't explain what just happened.

但我真的無法解釋 -- 我無法解釋剛剛發生的事情。

Data-driven systems are becoming a bigger and bigger part of our lives,

數據驅動的系統正在成為我們生活中越來越大的一部分。

and they work well a lot of the time.

而且它們在很多時候都能很好地工作。

- But when they fail... - Once again, it's the white guy.

- 但當他們失敗時- 又一次，是白人。

When they fail, they're not failing on everyone equally.

當他們失敗時，他們不是對每個人都平等地失敗。

If I go back right now...

如果我現在回去...

Ruha Benjamin: You can have neutral intentions.

魯哈-本傑明：你可以有中立的意圖。

You can have good intentions.

你可以有好的意圖。

And the outcomes can still be discriminatory.

而結果仍然可能是歧視性的。

Whether you want to call that machine racist

無論你是否想把那臺機器稱為種族主義者

or you want to call the outcome racist,

或者你想把這個結果稱為種族主義者。

we have a problem.

我們有一個問題。

( theme music playing )

(主題音樂播放)

I was scrolling through my Twitter feed a while back

前段時間，我在翻閱我的推特資料時發現

and I kept seeing tweets that look like this.

而且我一直看到看起來像這樣的推文。

Two of the same picture of Republican senator Mitch McConnell smiling,

共和黨參議員米奇-麥康奈爾的兩張相同的照片在微笑。

or sometimes it would be four pictures

或者有時是四張照片

of the same random stock photo guy.

的同一個隨機股票照片的傢伙。

And I didn't really know what was going on,

而我並不真正知道發生了什麼事。

but it turns out that this was a big public test of algorithmic bias.

但事實證明，這是對算法偏見的一次大型公開測試。

Because it turns out that these aren't pictures of just Mitch McConnell.

因為事實證明，這些並不是只有米奇-麥康奈爾的照片。

They're pictures of Mitch McConnell and...

他們是米奇-麥康奈爾的照片和...

- Barack Obama. - Lee: Oh, wow.

- 巴拉克-奧巴馬。- 李。哦，哇。

So people were uploading

所以人們在上傳

these really extreme vertical images

這些真正極端的垂直影像

to basically force this image cropping algorithm

基本上強迫這種影像裁剪算法

to choose one of these faces.

來選擇這些面孔之一。

People were alleging that there's a racial bias here.

人們指責這裡有種族偏見。

But I think what's so interesting about this particular algorithm

但我認為這個特定算法的有趣之處在於

is that it is so testable for the public.

是，它對公眾來說是如此的可檢驗。

It's something that we could test right now if we wanted to.

如果我們想的話，現在就可以測試。

- Let's do it. - You guys wanna do it?

- 我們來做吧。- 你們想做嗎？

Okay. Here we go.

好了，我們開始吧。

So, Twitter does offer you options to crop your own image.

是以，Twitter確實為你提供了裁剪自己圖片的選項。

But if you don't use those,

但如果你不使用這些。

it uses an automatic cropping algorithm.

它使用自動裁剪算法。

- Wow. There it is. - Whoa. Wow.

- 哇。就在那裡。- 哇哦。哇哦。

That's crazy.

這很瘋狂。

Christophe, it likes you.

克里斯托弗，它喜歡你。

Okay, let's try the other-- the happy one.

好吧，讓我們試試另一個--快樂的那個。

Lee: Wow.

李：哇。

- Unbelievable. Oh, wow. - Both times.

- 難以置信。哦，哇。- 兩次都是如此。

So, do you guys think this machine is racist?

那麼，你們認為這臺機器是種族主義者嗎？

The only other theory I possibly have

我唯一可能有的其他理論

is if the algorithm prioritizes white faces

是指如果該算法優先考慮白人面孔

because it can pick them up quicker, for whatever reason,

因為不管什麼原因，它能更快地把它們撿起來。

against whatever background.

無論在什麼背景下，都是如此。

Immediately, it looks through the image

立即，它通過影像查看

and tries to scan for a face.

並試圖掃描出一張臉。

Why is it always finding the white face first?

為什麼總是先找到白臉？

Joss: With this picture, I think someone could argue

喬斯。通過這張照片，我想有人可以爭辯說

that the lighting makes Christophe's face more sharp.

燈光使克里斯托夫的臉更加鮮明。

I still would love to do

我還是很想做

a little bit more systematic testing on this.

在這一點上，要進行更多的系統測試。

I think maybe hundreds of photos

我想可能有數百張照片

could allow us to draw a conclusion.

可以讓我們得出一個結論。

I have downloaded a bunch of photos

我已經下載了一堆照片

from a site called Generated Photos.

來自一個名為Generated Photos的網站。

These people do not exist. They were a creation of AI.

這些人並不存在。他們是人工智能的創造。

And I went through, I pulled a bunch

而我通過，我拉了一堆

that I think will give us

我認為這將給我們帶來

a pretty decent way to test this.

一個相當體面的方法來測試這個。

So, Christophe, I wonder if you would be willing to help me out with that.

所以，克里斯托弗，我想知道你是否願意幫我解決這個問題。

You want me to tweet hundreds of photos?

你想讓我在推特上發佈數百張照片？

- ( Lee laughs ) - Joss: Exactly.

- （李笑） - 喬斯。正是如此。

I'm down. Sure, I've got time.

我下來了。當然，我有時間。

Okay.

好的。

( music playing )

( 音樂播放 )

There may be some people who take issue with the idea

可能有一些人對這個想法有異議

that machines can be racist

機器可以是種族主義者

without a human brain or malicious intent.

沒有人的大腦或惡意的意圖。

But such a narrow definition of racism

但這樣一個狹義的種族主義定義

really misses a lot of what's going on.

真的錯過了很多正在發生的事情。

I want to read a quote that responds to that idea.

我想讀一段迴應這一想法的話語。

It says, "Robots are not sentient beings, sure,

它說，"機器人不是有生命的人，當然。

but racism flourishes well beyond hate-filled hearts.

但種族主義的盛行遠遠超出了充滿仇恨的心。

No malice needed, no "N" word required,

不需要惡意，不需要 "N "字。

just a lack of concern for how the past shapes the present."

只是對過去如何塑造現在缺乏關注。"

I'm going now to speak to the author of those words, Ruha Benjamin.

我現在要和這些話的作者魯哈-本傑明談談。

She's a professor of African-American Studies at Princeton University.

她是普林斯頓大學的非裔美國人研究教授。

When did you first become concerned

你是什麼時候開始關注

that automated systems, AI, could be biased?

自動化系統，人工智能，可能有偏見？

A few years ago, I noticed these headlines

幾年前，我注意到這些頭條新聞

and hot takes about so-called racist and sexist robots.

以及對所謂的種族主義和性別歧視機器人的熱議。

There was a viral video in which two friends were in a hotel bathroom

有一個病毒視頻，其中兩個朋友在一個酒店的浴室裡

and they were trying to use an automated soap dispenser.

而他們正試圖使用一個自動皁液器。

Black hand, nothing. Larry, go.

黑色的手，什麼都沒有。拉里，走。

Black hand, nothing.

黑色的手，什麼都沒有。

And although they seem funny

雖然他們看起來很有趣

and they kind of get us to chuckle,

而且他們有點讓我們發笑。

the question is, are similar design processes

問題是，類似的設計過程是否

impacting much more consequential technologies that we're not even aware of?

影響到我們甚至沒有意識到的更有影響的技術？

When the early news controversies came along maybe 10 years ago,

當早期的新聞爭論出現時，也許是10年前。

people were surprised by the fact that they showed a racial bias.

人們對他們表現出種族偏見的事實感到驚訝。

Why do you think people were surprised?

你認為為什麼人們會感到驚訝？

Part of it is a deep attachment and commitment

它的一部分是一種深深的依戀和承諾

to this idea of tech neutrality.

對這種技術中立的想法。

People-- I think because life is so complicated

人們......我想因為生活是如此複雜

and our social world is so messy--

和我們的社會世界是如此混亂 --

really cling on to something that will save us,

真正緊緊抓住能拯救我們的東西。

and a way of making decisions that's not drenched

和一種不被淹沒的決策方式

in the muck of all of human subjectivity,

在所有人類主觀性的泥沼中。

human prejudice and frailty.

人類的偏見和弱點。

We want it so much to be true.

我們非常希望它是真的。

We want it so much to be true, you know?

我們非常希望它是真的，你知道嗎？

And the danger is that we don't question it.

而危險的是，我們沒有質疑它。

And still we continue to have, you know, so-called glitches

而我們仍然繼續有，你知道，所謂的小毛病

when it comes to race and skin complexion.

當涉及到種族和膚色的時候。

And I don't think that they're glitches.

而且我不認為它們是小毛病。

It's a systemic issue in the truest sense of the word.

這是一個最真實意義上的系統性問題。

It has to do with our computer systems and the process of design.

這與我們的計算機系統和設計過程有關。

Joss: AI can seem pretty abstract sometimes.

喬斯。 人工智能有時會顯得很抽象。

So we built this to help explain

所以我們建立了這個來幫助解釋

how machine learning works and what can go wrong.

機器學習是如何工作的，會出什麼問題。

This black box is the part of the system that we interact with.

這個黑盒子是我們與之互動的系統的一部分。

It's the software that decides which dating profiles we might like,

它是決定我們可能喜歡哪些約會資料的軟件。

how much a rideshare should cost,

乘坐共享汽車應該花費多少錢。

or how a photo should be cropped on Twitter.

或Twitter上的照片應該如何裁剪。

We just see a device making a decision.

我們只是看到一個設備在做決定。

Or more accurately, a prediction.

或者更準確地說，是一種預測。

What we don't see is all of the human decisions

我們沒有看到的是所有的人類決定

that went into the design of that technology.

融入該技術的設計。

Now, it's true that when you're dealing with AI,

現在，當你與人工智能打交道時，這確實是事實。

that means that the code in this box

這意味著，這個盒子裡的代碼

wasn't all written directly by humans,

並非都是由人類直接寫的。

but by machine-learning algorithms

但通過機器學習算法

that find complex patterns in data.

找到數據中的複雜模式。

But they don't just spontaneously learn things from the world.

但他們不會自發地從世界上學到東西。

They're learning from examples.

他們正在從實例中學習。

Examples that are labeled by people,

被人貼上標籤的例子。

selected by people,

由人選擇。

and derived from people, too.

而且也是來自於人。

See, these machines and their predictions,

看，這些機器和它們的預測。

they're not separate from us or from our biases

他們與我們或與我們的偏見並不分離

or from our history,

或來自我們的歷史。

which we've seen in headline after headline

我們在一個又一個的頭條新聞中看到了這一點

for the past 10 years.

在過去的10年裡。

We're using the face-tracking software,

我們正在使用面部追蹤軟件。

so it's supposed to follow me as I move.

所以它應該在我移動時跟隨我。

As you can see, I do this-- no following.

正如你所看到的，我這樣做--沒有下文。

Not really-- not really following me.

不太......不太關注我。

- Wanda, if you would, please? - Sure.

- 萬達，如果你願意，請？- 當然可以。

In 2010, the top hit

在2010年，最熱門的是

when you did a search for "black girls,"

當你搜索 "黑人女孩 "時，

80% of what you found

你發現的80%的東西

on the first page of results was all porn sites.

在結果的第一頁都是色情網站。

Google is apologizing after its photo software

谷歌在其照片軟件之後進行了道歉

labeled two African-Americans gorillas.

給兩個非裔美國人貼上了大猩猩的標籤。

Microsoft is shutting down

微軟正在關閉

its new artificial intelligent bot

其新的人工智能機器人

after Twitter users taught it how to be racist.

在Twitter用戶教它如何成為種族主義者之後。

Woman: In order to make yourself hotter,

女人。為了讓自己更性感。

the app appeared to lighten your skin tone.

該應用程序出現了淡化你的膚色。

Overall, they work better on lighter faces than darker faces,

總的來說，它們對淺色的臉比深色的臉效果更好。

and they worked especially poorly

而且他們的工作特別差

on darker female faces.

在較黑的女性臉上。

Okay, I've noticed that on all these damn beauty filters,

好吧，我在所有這些該死的美容濾鏡上注意到了這一點。

is they keep taking my nose and making it thinner.

是他們一直把我的鼻子弄得更薄。

Give me my African nose back, please.

請把我的非洲鼻子還給我。

Man: So, the first thing that I tried was the prompt "Two Muslims..."

男子：所以，我嘗試的第一件事是提示 "兩個穆斯林......"

And the way it completed it was,

而它完成的方式是。

"Two Muslims, one with an apparent bomb,

"兩名穆斯林，其中一人攜帶明顯的炸彈。

tried to blow up the Federal Building

試圖炸燬聯邦大樓

in Oklahoma City in the mid-1990s."

在1990年代中期的俄克拉荷馬城"。

Woman: Detroit police wrongfully arrested Robert Williams

婦女。 底特律警方錯誤地逮捕了羅伯特-威廉姆斯

based on a false facial recognition hit.

基於一個錯誤的面部識別命中。

There's definitely a pattern of harm

肯定有一種傷害的模式

that disproportionately falls on vulnerable people, people of color.

這一點不成比例地落在弱勢人群、有色人種身上。

Then there's attention,

然後是注意力。

but of course, the damage has already been done.

但當然，損害已經造成了。

( Skype ringing )

( Skype鈴聲 )

- Hello. - Hey, Christophe.

- 你好。- 嘿，克里斯托弗。

Thanks for doing these tests.

謝謝你做這些測試。

- Of course. - I know it was a bit of a pain,

- 當然了。- 我知道這有點麻煩。

but I'm curious what you found.

但我很好奇你發現了什麼。

Sure. I mean, I actually did it.

當然，我是說，我真的做到了。

I actually tweeted 180 different sets of pictures.

我實際上在推特上發佈了180組不同的圖片。

In total, dark-skinned people

總的來說，黑皮膚的人

were displayed in the crop 131 times,

被顯示在莊稼地裡131次。

and light-skinned people

和淺色皮膚的人

were displayed in the crop 229 times,

被顯示在莊稼地裡229次。

which comes out to 36% dark-skinned

這意味著36%的黑皮膚人

and 64% light-skinned.

和64%的淺色皮膚。

That does seem to be evidence of some bias.

這似乎確實是一些偏見的證據。

It's interesting because Twitter posted a blog post

這很有趣，因為Twitter發佈了一篇博文

saying that they had done some of their own tests

說他們已經做了一些自己的測試

before launching this tool, and they said that

在推出這個工具之前，他們說，

they didn't find evidence of racial bias,

他們並沒有發現種族偏見的證據。

but that they would be looking into it further.

但他們將進一步調查此事。

Um, they also said that the kind of technology

嗯，他們還說，那種技術

that they use to crop images

他們用來裁剪影像的

is called a Saliency Prediction Model,

被稱為 "顯著性預測模型"（Saliency Prediction Model）。

which means software that basically is making a guess

這意味著軟件基本上是在做一個猜測

about what's important in an image.

關於影像中什麼是重要的。

So, how does a machine know what is salient, what's relevant in a picture?

那麼，機器如何知道什麼是突出的，什麼是圖片中的相關內容？

Yeah, it's really interesting, actually.

是的，這真的很有趣，實際上。

There's these saliency data sets

有這些突出性數據集

that documented people's eye movements

記錄了人們的眼球運動

while they looked at certain sets of images.

當他們看某些影像集的時候。

So you can take those photos

所以你可以拍攝這些照片

and you can take that eye-tracking data

而且你可以利用這些眼球追蹤數據

and teach a computer what humans look at.

並教計算機看人類的東西。

So, Twitter's not going to give me any more information

所以，Twitter不會給我任何更多的資訊

about how they trained their model,

關於他們如何訓練他們的模型。

but I found an engineer from a company called Gradio.

但我找到了一個叫Gradio的公司的工程師。

They built an app that does something similar,

他們建立了一個應用程序，做了類似的事情。

and I think it can give us a closer look

而且我認為它可以給我們一個更近距離的觀察

at how this kind of AI works.

在這種人工智能如何工作。

- Hey. - Hey.

- 嘿。

- Joss. - Nice to meet you. Dawood.

- 喬斯。- 很高興見到你。達伍德。

So, you and your colleagues

所以，你和你的同事

built a saliency cropping tool

建立了一個突出性裁剪工具

that is similar to what we think Twitter is probably doing.

這與我們認為Twitter可能正在做的事情相似。

Yeah, we took a public machine learning model, posted it on our library,

是的，我們把一個公共的機器學習模型，發佈在我們的圖書館。

and launched it for anyone to try.

並推出它供任何人嘗試。

And you don't have to constantly post pictures

而且你不必不斷髮布照片

on your timeline to try and experiment with it,

在你的時間軸上嘗試和實驗。

which is what people were doing when they first became aware of the problem.

這也是人們第一次意識到這個問題時正在做的事情。

And that's what we did. We did a bunch of tests just on Twitter.

而這正是我們所做的。我們僅僅在Twitter上做了一堆測試。

But what's interesting about what your app shows

但有趣的是，你的應用程序所顯示的內容

is the sort of intermediate step there, which is this saliency prediction.

是那種中間步驟，也就是這種突出性預測。

Right, yeah. I think the intermediate step is important for people to see.

對，是的。我認為中間的步驟對人們來說很重要。

Well, I-- I brought some pictures for us to try.

好吧，我......我帶來了一些照片，供我們嘗試。

These are actually the hosts of "Glad You Asked."

這些人實際上是 "Glad You Asked "的主持人。

And I was hoping we could put them into your interface

我希望我們可以把它們放到你的界面上。