Name: 大型語言模型介紹 (Introduction to Large Language Models)
Uploaded: 2024-07-27T08:46:23.000Z
Duration: 16 min 11 s

未來進行式

Today I'm going to be talking about large language models.

今天我要說的是大型語言模型。

I'm a customer engineer here at Google Cloud, and today I'm going to teach you everything you need to know about LLMs.

我是 Google Cloud 的客戶工程師，今天我將向你傳授關於 LLM 的一切知識。

In this course, you're going to learn to define large language models, describe LLM use cases, explain prompt tuning, and describe Google's generative AI development tools.

在本課程中，您將學習如何定義大型語言模型、描述 LLM 用例、解釋提示調整並介紹 Google 的生成式人工智能開發工具。

Large language models, or LLMs, are a subset of deep learning.

大型語言模型或 LLM 是深度學習的一個子集。

To find out more about deep learning, check out our Introduction to Generative AI course video.

要了解有關深度學習的更多資訊，請觀看我們的 "生成式人工智能入門 "課程視頻。

LLMs and generative AI intersect and they are both a part of deep learning.

LLM 和生成式人工智能相互交叉，它們都是深度學習的一部分。

Another area of AI you may be hearing a lot about is generative AI.

您可能經常聽到的另一個人工智能領域是生成式人工智能。

This is a type of artificial intelligence that can produce new content including text, images, audio, and synthetic data.

這是一種人工智能，可以生成新的內容，包括文本、影像、音頻和合成數據。

那麼，什麼是大型語言模型呢？

Large language models refer to large, general purpose language models that can be pre-trained and then fine-tuned for specific purposes.

大型語言模型指的是大型通用語言模型，可以進行預訓練，然後針對特定目的進行微調。

Often you train your dog basic commands such as sit, come, down, and stay.

通常情況下，您會訓練狗狗基本的指令，如坐、來、趴下和停留。

These commands are normally sufficient for everyday life and help your dog become a good canine citizen.

這些指令通常足以應付日常生活，並能幫助您的愛犬成為良好的犬類公民。

But if you need special service dogs such as a police dog, a guide dog, or a hunting dog, you add special trainings, right?

但如果你需要特殊的服務犬，如警犬、導盲犬或獵犬，你就需要增加特殊訓練，對嗎？

A similar idea applies to large language models.

類似的想法也適用於大型語言模型。

These models are trained for general purposes to solve common language problems such as text classification, question answering, document summarization, and text generation across industries.

這些模型經過訓練後可用於解決常見的語言問題，如文本分類、問題解答、文檔摘要和跨行業文本生成。

The models can then be tailored to solve specific problems in different fields such as retail, finance, and entertainment using a relatively small size of field datasets.

然後，就可以利用相對較小的現場數據集對這些模型進行定製，以解決零售、金融和娛樂等不同領域的具體問題。

So now that you've got that down, let's further break down the concept into three major features of large language models.

既然你已經明白了這一點，讓我們進一步將這一概念分解為大型語言模型的三大特徵。

First is the enormous size of the training dataset, sometimes at the petabyte scale.

首先是訓練數據集的巨大規模，有時甚至達到 PB 級。

Second, it refers to the parameter count.

In machine learning, parameters are often called hyperparameters.

在機器學習中，參數通常被稱為超參數。

Parameters are basically the memories and the knowledge the machine learned from the model training.

參數基本上是機器從模型訓練中學到的記憶和知識。

Parameters define the skill of a model in solving a problem such as predicting text.

參數定義了模型解決文本預測等問題的技能。

General purpose is when the models are sufficient to solve common problems.

通用型是指模型足以解決常見問題。

First is the commonality of human language regardless of the specific tasks.

首先是人類語言的共性，無論具體任務如何。

Only certain organizations have the capability to train such large language models with huge datasets and a tremendous number of parameters.

只有某些機構有能力利用龐大的數據集和大量的參數來訓練這種大型語言模型。

How about letting them create fundamental language models for others to use?

讓他們創建基本語言模型供他人使用如何？

So this leaves us with our last terms, pre-trained and fine-tuned, which mean to pre-train a large model for a general purpose with a large dataset and then fine-tune it for specific aims with a much smaller dataset.

這就剩下了最後一個術語，預訓練和微調，這意味著使用大型數據集為通用目的預訓練一個大型模型，然後使用小得多的數據集為特定目的對其進行微調。

So now that we've nailed down the definition of what large language models LLMs are, we can move on to describing LLM use cases.

既然我們已經明確了大型語言模型 LLM 的定義，那麼接下來就可以描述 LLM 的使用案例了。

The benefits of using large language models are straightforward.

使用大型語言模型的好處顯而易見。

First, a single model can be used for different tasks.

首先，一個模型可用於不同的任務。

These large language models that are trained with petabytes of data and generate billions of parameters are smart enough to solve different tasks, including language translation, sentence completion, text classification, question answering, and more.

這些大型語言模型使用 PB 級數據進行訓練，可生成數十億個參數，足以解決不同的任務，包括語言翻譯、句子補全、文本分類、問題解答等。

Second, large language models require minimal field training data when you tailor them to solve a specific problem.

其次，大型語言模型為解決特定問題而量身定製時，需要的現場訓練數據極少。

Large language models obtain decent performance even with little domain training data.

即使只有很少的領域訓練數據，大型語言模型也能獲得不錯的性能。

In other words, they can be used for few-shot or even zero-shot scenarios.

換句話說，它們可以用於少發甚至零發的情況。

In machine learning, few-shot refers to training a model with minimal data, and zero-shot implies that a model can recognize things that have not explicitly been taught in the training before.

在機器學習中，"少數據 "指的是用最少的數據來訓練模型，而 "零數據 "指的是模型能夠識別之前訓練中沒有明確教授的事物。

Third, the performance of large language models is continuously growing when you add more data and parameters.

第三，當您添加更多數據和參數時，大型語言模型的性能會不斷提高。

In April 2022, Google released POM, short for Pathways Language Model, a 540 billion parameter model that achieves a state-of-the-art performance across multiple language tasks.

2022 年 4 月，谷歌發佈了 POM（Pathways Language Model 的縮寫），這是一個 5400 億參數的模型，在多個語言任務中實現了最先進的性能。

POM is a dense decoder-only transformer model.

POM 是一個僅有密集解碼器的變壓器模型。

It leverages a new pathway system which enabled Google to efficiently train a single model across multiple TPU v4 pods.

它利用新的路徑系統，使谷歌能夠在多個 TPU v4 pod 上高效地訓練一個模型。

Pathways is a new AI architecture that will handle many tasks at once, learn new tasks quickly, and reflect a better understanding of the world.

Pathways 是一種全新的人工智能架構，可以同時處理多項任務，快速學習新任務，並反映出對世界的更好理解。

The system enables POM to orchestrate distributed computation for accelerators, but I'm getting ahead of myself.

該系統使 POM 能夠為加速器協調分佈式計算，但我說得太多了。

I previously mentioned that POM is a transformer model.

我之前提到過，POM 是一種變壓器模型。

讓我來解釋一下這意味著什麼。

A transformer model consists of an encoder and a decoder.

變壓器模型由編碼器和解碼器組成。

The encoder encodes the input sequence and passes it to the decoder, which learns how to decode the representations for a relevant task.

編碼器對輸入序列進行編碼，然後將其傳遞給解碼器，解碼器學習如何對相關任務的表徵進行解碼。

We've come a long way from traditional programming to neural networks to generative models.

從傳統編程到神經網絡，再到生成模型，我們已經走過了漫長的道路。

In traditional programming, we used to have to hard code the rules for distinguishing a cat.

在傳統編程中，我們過去不得不硬編碼區分貓的規則。

Type, animal, legs 4, ears 2, fur yes, likes, yarn and catnip.

類型，動物，4 條腿，2 只耳朵，有毛，喜歡毛線和貓薄荷。

In the wave of neural networks, we could give the network pictures of cats and dogs and ask, is this a cat?

在神經網絡浪潮中，我們可以給網絡提供貓和狗的圖片，然後問：這是貓嗎？

What's really cool is that in the generative wave, we as users can generate our own content, whether it be text, images, audio, video, or more.

最酷的是，在生成浪潮中，作為用戶，我們可以生成自己的內容，無論是文字、圖片、音頻、視頻還是其他內容。

For example, models like POM, or pathways language model, or Lambda, language model for dialogue applications, ingest very, very large data from multiple sources across the internet, and build foundation language models we can use simply by asking a question, whether typing it into a prompt or verbally talking into the prompt itself.

例如，像 POM 這樣的模型，或路徑語言模型，或用於對話應用的語言模型 Lambda，可以從互聯網上的多個來源獲取非常非常龐大的數據，並建立基礎語言模型，我們只需提出一個問題，無論是將問題輸入到提示符中，還是對著提示符本身進行口頭交談，就可以使用這些模型。

It can give you everything it has learned about a cat.

它可以向你提供關於貓的一切資訊。

Let's compare LLM development using pre-trained models with traditional ML development.

讓我們將使用預訓練模型的 LLM 開發與傳統的 ML 開發進行比較。

First, with LLM development, you don't need to be an expert.

首先，在 LLM 開發方面，您不需要成為專家。

You don't need training examples, and there is no need to train a model.

您不需要訓練示例，也不需要訓練模型。

All you need to do is think about prompt design, which is a process of creating a prompt that is clear, concise, and informative.

您所需要做的就是思考提示語的設計，這是一個創建清晰、簡潔、資訊豐富的提示語的過程。

It is an important part of natural language processing, or NLP for short.

它是自然語言處理（簡稱 NLP）的重要組成部分。

In traditional machine learning, you need expertise, training examples, compute time, and hardware.

在傳統的機器學習中，你需要專業知識、訓練示例、計算時間和硬件。

That's a lot more requirements than LLM development.

這比法律碩士的發展要求要高得多。

Let's take a look at an example of a text generation use case to really drive the point home.

讓我們來看一個文本生成使用案例的例子，以便真正理解這一點。

Question answering, or QA, is a subfield of natural language processing that deals with the task of automatically answering questions posed in natural language.

問題解答或 QA 是自然語言處理的一個子領域，主要處理自動回答用自然語言提出的問題的任務。

QA systems are typically trained on a large amount of text and code, and they are able to answer a wide range of questions, including factual, definitional, and opinion-based questions.

質量保證系統通常會在大量文本和代碼的基礎上進行訓練，能夠回答各種問題，包括事實性問題、定義性問題和觀點性問題。

The key here is that you needed domain knowledge to develop these question answering models.

關鍵在於，開發這些問題解答模型需要領域知識。

Let's make this clear with a real-world example.

讓我們用一個真實的例子來說明這一點。

Domain knowledge is required to develop a question answering model for customer IT support, or healthcare, or supply chain.

開發客戶 IT 支持、醫療保健或供應鏈問題解答模型需要領域知識。

But using generative QA, the model generates free text directly based on the context.

但使用生成式質量保證時，模型會根據上下文直接生成自由文本。

Let me show you a few examples of how cool this is.

Let's look at three questions given to Gemini, a large language model chatbot developed by Google AI.

讓我們來看看由谷歌人工智能開發的大型語言模型哈拉機器人雙子座提出的三個問題。

Gemini first shares how net profit is calculated, then performs the calculation.

雙子座首先介紹淨利潤的計算方法，然後進行計算。

Then Gemini provides the definition of net profit.

然後，雙子座提供了淨利潤的定義。