Name: AI 架構是什麼？LLM、RAG 和 AI 硬體一次搞懂！ (What Is an AI Stack? LLMs, RAG, & AI Hardware)
Uploaded: 2025-11-10T00:30:50.000Z
Duration: 9 min 6 s

程度副詞

Whether you're building an experimental prototype for your own personal use, or creating an application to power an entire organization, there are key components of the AI technology stack that you must get right to build AI systems that can do more than just generate answers, but

不論你是要為個人用途打造實驗性原型，還是要建立一個能為整個組織提供動力的應用程式，AI 技術堆疊中都有一些關鍵元件是必須掌握好的，這樣才能建立出不只產生答案，更能

解決真正有意義的問題的 AI 系統。

Say for instance, I'm building an AI powered application to help drug discovery researchers understand and analyze the latest scientific papers in their domain.

假設我正在建立一個 AI 驅動的應用程式，來協助藥物發現的研究人員理解和分析他們領域內的最新科學論文。

Maybe it starts with a model that I recently heard about that is supposed to be better at highly complex tasks like that of a PhD researcher.

也許它會從一個我最近聽說的模型開始，這個模型據說在像博士研究員那樣的高度複雜任務上表現更好。

Model is an important layer of the stack, but it's just one piece of the puzzle.

模型是堆疊中一個重要的層級，但它只是拼圖中的一塊。

There's also the infrastructure that that model will run on, because not all LLMs, large language models can run on standard and enterprise CPU based servers, and

還有模型將在其上運行的基礎設施，因為並非所有大型語言模型都能在標準和企業級 CPU 伺服器上運行，而且

not all are small enough to run on a laptop.

並非所有模型都小到可以在筆記型電腦上運行。

So it matters what infrastructure you have access to and how you choose to deploy it.

所以，你擁有的基礎設施以及你如何選擇部署它，都是很重要的。

Next is data, because in this example, the whole point is to help scientists understand the latest papers in their field.

接下來是資料，因為在這個例子中，重點是協助科學家理解他們領域內的最新論文。

And models typically have a knowledge cutoff date.

而且模型通常會有一個知識截止日期。

So if we want to talk about papers from, say the past three months, that means we have to provide the AI system with extra data.

所以如果我們要討論像是過去三個月內的論文，那就代表我們必須提供 AI 系統額外的資料。

Next would be the orchestration layer, because to do a complex task like this probably is going to require more than simply providing a large prompt into the AI system and getting a single output out.

接下來是協調層，因為要完成像這樣複雜的任務，可能不只是提供一個大型提示給 AI 系統然後得到一個輸出就夠了。

Instead, we'll want to break that user query up into different parts, help plan how the AI solution is going to actually tackle this problem, what data it needs, and

取而代之的是，我們要將使用者的查詢分解成不同的部分，協助規劃 AI 解決方案將如何實際處理這個問題，它需要什麼資料，以及

then do the summarization and creating an answer and maybe even review that answer.

然後進行摘要並創建答案，甚至可能審查該答案。

And this is because at the end of the day, there's a user using this tool.

這是因為歸根究柢，會有一個使用者在使用這個工具。

So there will have to be an interface that defines what the inputs will be and what the outputs will be.

所以必須有一個介面來定義輸入是什麼以及輸出是什麼。

It it might not be as simple as text in and text out.

它可能不像文字輸入和文字輸出那麼簡單。

And there's also the issue of integrations.

So will the actual results of this be something that's integrated into other tools that this user uses?

所以實際的結果會不會是整合到使用者使用的其他工具裡？

It's important to understand all the layers of the AI stack, whether you're building a solution from scratch or using solutions which might manage several of these layers for you as a service.

瞭解 AI 技術堆疊的所有層級都很重要，無論你是從頭開始建構 AI 解決方案，還是使用可能為你提供多個層級服務的解決方案。

This is because across the stack, from the hardware all the way up to the user interface level, the choices you make will have important implications on your solution's quality, its speed, its cost, and

這是因為從硬體到使用者介面層級，你在整個技術堆疊中所做的選擇，都會對你解決方案的品質、速度、成本以及

When it comes to infrastructure, LLMs generally require AI specific hardware, specifically GPUs, and

在基礎設施方面，大型語言模型通常需要專門的 AI 硬體，特別是 GPU，

these can be deployed in one of three ways.

而這些可以透過三種方式部署。

第一種是地端部署，也就是說，

assuming you have the means and resource to buy this kind of infrastructure yourself.

假設你有能力和資源自己購買這類基礎設施。

Second option would be cloud and that would allow you to rent this capacity and be able to scale it up or down as needed.

第二種是雲端，這能讓你租用這些運算能力，並根據需要進行擴展或縮減。

Finally would be local, which usually means on your laptop.

最後一種是在地部署，這通常是指在你的筆記型電腦上。

Not all laptops can support LLMs of different sizes, but

並非所有筆記型電腦都能支援不同大小的 大型語言模型，但

there are certainly LLMs on the smaller end of the range that can be run on the kind of GPUs available in a standard laptop.

確實有一些較小型的 大型語言模型，可以在一般筆記型電腦的 GPU 上運行。

The next layer is models, so AI builders have plenty of choice when it comes to what model they can use.

下一層是模型，所以 AI 開發者在選擇要使用的模型時，有很多選擇。

One dimension to consider is whether the model is open versus proprietary.

其中一個考量面向是模型是開源的還是專有的。

We also have small language models that might be lighter weight and able to fit on more lightweight hardware, but

我們也有小型語言模型，它們可能更輕巧，能裝載在更輕量的硬體上，但

might not have exactly the same thinking capacity as a large language model and instead be specialized for more specific things.

可能沒有和大型語言模型完全相同的思考能力，而是專門用於更具體的事務。

Finally is specialization, which sometimes goes hand in hand with size.

最後是專精化，這有時與大小是相輔相成的。

Some models might perform better on things like reasoning or tool calling or generating code.

有些模型在推理、工具調用或生成程式碼等方面可能表現得更好。

Others might have different language strengths than others.

有些模型在語言上的優勢會跟其他模型不一樣。

There are plenty of new models, over 2 million already in model catalogs like Hugging Face that can serve any mix of these different needs that an AI builder might have.

模型庫裡有非常多新模型，像是 Hugging Face 上就已經有超過兩百萬個模型，可以滿足 AI 開發者各種不同的需求。

技術堆疊的下一個層級是資料。

This breaks up into a few different components, so the first would be data sources themselves to supplement the model's knowledge.

這可以抽成幾個不同的部分，第一個是資料來源本身，用來補充模型的知識。

This could also include the pipelines to do any processing, pre processing, post processing of that data, as well as vector databases you may use or retrieval systems.

這也可能包含處理、預處理、後處理這些資料的管線，以及你可能會使用的向量資料庫或檢索系統。

Also known as RAG vector databases is the step where that external data is actually vectorized into embeddings that are saved so your model can retrieve that context more quickly and augment it with this additional knowledge that the base model does not have.

向量資料庫，也就是 RAG，這個步驟會將外部資料向量化成嵌入式，然後儲存起來，這樣你的模型就能更快地檢索到這些內容，並用模型本身沒有的這些額外知識來增強它。

That's important because base models are usually trained on publicly available information which might not always be complete.

這很重要，因為基礎模型通常是根據公開資訊訓練的，而這些資訊不一定總是完整的。

To accomplish the task that you have, you might need to supplement with additional data.

為了完成你手上的任務，你可能需要補充額外的資料。

The next layer is orchestration, because building an AI system that does something more complex than just generating text or answering questions requires breaking the initial user input down into smaller tasks.

下一個層級是協調，因為要建立一個比單純生成文字或回答問題更複雜的 AI 系統，就需要將使用者最初的輸入分解成更小的任務。

Those can start with things like thinking, using the model's reasoning ability to plan out how it will tackle the problem.

這些任務可以從像是思考開始，利用模型的推理能力來規劃如何解決問題。

That can also include things like execution where the model does tool calling or function calling, as well as steps like reviewing, where an LLM can actually provide its own critique of the initial generated responses and initiate feedback loops to even improve those responses responses.

這也可以包含像是執行，模型會進行工具呼叫或函式呼叫，以及審閱步驟，大型語言模型可以實際提供對初始生成回應的批評，並啟動回饋循環來進一步改進這些回應。

This layer is very quickly evolving with new protocols like MCP and new architectures for how to best orchestrate increasingly complex tasks.

這個層級正快速演進，有新的協定像是 MCP，以及用於如何最佳協調日益複雜任務的新架構。

So the most widely used AI systems do follow a pretty simple design of text in and text out.

所以最廣泛使用的 AI 系統都遵循一個相當簡單的設計，也就是輸入文字，輸出文字。

But as we use these tools in our work and life, there are important features that become critical for the actual usability of AI.

但當我們在工作和生活中使用這些工具時，有些重要功能對於 AI 的實際可用性變得至關重要。

And these factors make up the application layer.

而這些因素構成了應用程式層。

The most classic interface is text in and text out.

最經典的介面就是輸入文字，輸出文字。

But there are other modalities that can be very valuable for certain tasks too, like image, audio, numerical data sets, and

但對於某些任務，其他模態也非常有價值，像是圖像、音訊、數值資料集，以及

Also in the interface, it's really important to keep in mind the ability to do things like revisions or citations, so that when the user sees what the model comes up with, they have the ability to edit that or inquire on it further.

此外，在介面方面，我們真的需要記住能夠進行修訂或引用等功能，這樣當使用者看到模型產生的結果時，他們就能夠編輯或進一步詢問。

The second consideration is integrations and and that comes both in the form of integrations of allowing other tools that the user uses to actually send inputs to the AI system, or to take the model outputs and automate how that gets integrated into some of the tools that they use in their day to day work.

第二個考量是整合，這包括允許使用者使用的其他工具實際輸入到AI系統，或是將模型輸出整合到他們日常工作中使用的工具裡。

Altogether, these layers of the AI stack, from the hardware to the models, the data you use, how you orchestrate it, and

總而言之，AI技術堆疊的這些層級，從硬體到模型、使用的資料、協調方式，

the application and the usability of it matter, because when we have a clear understanding of how they fit together, we can see what's truly possible and make practical choices to design AI systems that are reliable, effective and aligned to our real world needs.

還有應用程式和可用性都很重要，因為當我們清楚瞭解它們如何結合時，就能看到什麼是真正可能的，並做出實際的選擇來設計可靠、有效且符合我們真實世界需求的AI系統。