It went viral, but not in the way they'd hoped.

這款產生器一夕爆紅，但不是因為他們期望的原因。

James Clayton picks up the story.

James Clayton 為您報導。

Yo, boys, I may need some help.

兄弟們，我這邊需要幫助。

I found a room with a bunch of mob spawners.

我發現了一個房間，裡面有一堆生怪磚。

You ever wondered what it would be like to be someone else?

你有想像過以別人的身份生活是怎樣的嗎？

With ElevenLabs AI technology, you can find out.

有了 ElevenLabs 的 AI 科技，你可以實現想像。

Given a minute of anyone's voice, the software can quickly put together an imitation.

只要喂這個 AI 軟體某人一分鐘的音頻，它就可以迅速模仿他的聲音。

Within days, people were imitating celebrity voices, making them say things they never said.

發佈幾天之內，人們已開始仿制名人的聲音，讓他們說出從沒說過的話。

I just think I would love to get Ratatouilled.

我超想要料理鼠王。

Ratatouilled?

料理鼠王？

Like, have a little guy up there.

對啊，養隻小傢伙在頭上。

Excuse me?

什麼？

You know, making me cook delicious meals.

你知道的，讓牠控制我煮好吃的菜。

This is the point where I tell you that you're actually listening to an AI voice,

現在正是時候讓我告訴你，實際上你一直在聽的這個聲音是 AI 仿音，

but it wasn't exactly the most convincing voice.

但這還不是最有說服力的聲音。

This is it.

這是音檔。

Testing out my voice clone.

聲音克隆測試。

I don't think it sounds like me that much, but what do you think?

我覺得聽起來沒有很像，你覺得呢？

I'm gonna record here too, just to, you know, to help them off.

我這裡也要錄，幫一下他們。

My producer, Ben, and I spoke to the CEO and co-founder of ElevenLabs, Matty Staniszewski, for the company's first broadcast interview.

我的製作人 Ben 和我採訪了 ElevenLabs 的執行長兼共同創辦人 Matty Staniszewski 進行了該公司首次影像採訪。

So I had a go at mimicking my own voice,

所以我試著仿制了我的聲音。

and it, you know, it was sort of reasonably successful,

算是有成功，

although it seemed to have a bit of an Australian accent.

雖然好像有點澳洲腔。

Majority of our work is based on public domain data, where majority of the voices and the work that we have are actually US American accents.

我們大部分數據取自公眾領域，其中絕大多數都是美國口音。

So if Ben did it, it might be more successful.

所以如果 Ben 去試，可能會更成功。

Ben should definitely try it.

對，Ben 應該去試試。

The company's terms and conditions are clear that you can only clone your own voice, but not everyone has abided by its rules.

該公司的條款寫得很清楚，只能克隆自己的聲音，但不是每個人都守規則。

On sites like 4chan, it's incredibly easy to find people who've taken celebrity voices and made them say racist and hateful things.

在像 4chan 這種網站上，很容易就可以找到用名人聲音生成的種族歧視和仇恨言論內容。

ElevenLabs accepts its platform has been used for hate speech.

ElevenLabs 坦然面對了其平臺被用於仇恨言論這件事。

You know, 99% of content is kind of positive that we've seen

你知道，我們所看到的 99% 內容都是正面內容，

and for less than 1% of what was generated, it was usually against terms of service,

只有不到 1% 的內容是違反平台條款的，

whether it was hate speech or variations of that.

無論是仇恨言論，還是其他相關內容。

The company says only paid users can clone voices now and they're developing a new tool to track down people abusing its platform.

該公司表示，現在只有付費用戶可以克隆聲音，同時他們也正在開發新工具來追蹤濫用此功能的人。

But ElevenLabs doesn't just do cloned voices.

但其實 ElevenLabs 的業務不只是訪制聲音。

Just give me a second.

等我一下喔。

The company has another tool where users can create a unique voice completely from scratch.

該公司還有一個工具，能讓用戶從頭生成獨創的語音。

Let's use a female gender and I'm gonna do a middle age.

我們選女性，中年。

And I think even more fitting with a British accent.

我想英國腔會不錯。

And I'm just gonna click generate.

點擊生成。

That's gonna create that entirely new voice that doesn't exist.

全新的語音就產生了。

Britain's first live public broadcast was made from the factory of Marconi's Wireless telegraph company in Chelmsford.

英國第一次公開直播是在馬可尼無線電報公司位於切姆斯福德的工廠裡進行的。

It's never been easier to imitate a politician's voice.

模仿政治人物的聲音從未如此簡單。

Sleepy Joe, you have run this country to its absolute ground.

愛睏喬，你把這個國家經營得一塌糊塗。

Well, at least I do run sometimes, if you know what I mean?

至少我有時真的會跑步，你有懂嗎？

Mark Nitzberg is a professor at UC Berkeley's AI research lab.

Mark Nitzberg 是加州大學伯克萊分校 AI 研究實驗室的教授。

I asked him whether we should be worried about the presidential elections in 2024.

我問他，該不該擔心 2024 年的總統選舉。

We'll very likely see many more deep fakes than we did in the last cycle just because it's so much simpler

我們很可能會面臨比上次選舉更多的深度造假，因為造假更簡單了。

and I'm not sure what measures we are taking to counter what effect that might have.

我不確定我們要採取什麼措施來應對可能產生的影響。

There are plenty of other concerns around this tech too.

關於這項技術，還有很多其他的擔憂。

For example, scams,

例如，詐騙，

this could make them far more believable and far easier to execute.

AI 能提升其可信度，並減低執行難度。

The Federal Trade Commission put out a statement in March,

聯邦貿易委員會於三月發表聲明，

warning people that scams imitating relatives can now be incredibly convincing.

警示人們現今假裝親屬聲音的騙局中，其以假亂真的程度令人髮指。

ElevenLabs themselves say that as AI develops,

ElevenLabs 也表示，

people are simply going to have to learn to trust voices they hear and things that they see less.

隨著人工智能發展，人們將不得不提高警覺，學會懷疑聽到及看到的東西。

A kind of the wider education is just making sure that people across know that

我們應更廣泛的教育人們，確保大家知道

as we think about audio, as we think about text, as we think about images,

我們聽到看到的音頻、文字及影像，

a lot of the content that has been created is likely or could be AI generated.

這些內容都可能是人工智能產生的。

And it's important to know that this might be the case.

而重要的是要知道，這可能是，這可能是情況。

You know, one needs to again take this approach where we think about we want in a society before we allow the release of these tools.

在允許這些工具上市之前，我們要考慮社會的需求是什麼。

Voice cloning for the public is here to stay.

但語音仿製是不會消失的。

It's very difficult to see how this is going to be put back in its box now.

現階段來看，不太可能回到之前的樣子。

But this is clearly disruptive technology that can be used for both good and bad.

知道的是，這項顛覆性技術顯然是把雙面刃。