Name: 針對人工智能後端使用案例的高級網卡模擬 (Advanced NIC simulation for AI back end use cases)
Uploaded: 2025-05-08T05:10:50.000Z
Duration: 22 min 14 s

我是漫威公司的高級首席工程師。

Today I'm gonna talk about advanced NIC simulation for AI backend use cases.

今天，我將談談針對人工智能後端用例的高級網卡模擬。

So this is a collaborative work with our team and I would like to acknowledge my team, VSR, Pradeep and Radha.

是以，這是與我們團隊的合作成果，我要感謝我的團隊 VSR、Pradeep 和 Radha。

So Sataranth, the co-presenter today along with me, couldn't make it, so I'll handle the whole presentation.

今天和我一起主持會議的薩塔蘭特來不了，所以由我來主持整個會議。

First I'll start with why we are doing this work, like why we need advanced NIC modeling and why is it particularly important for AI workloads.

首先，我要談談我們為什麼要做這項工作，比如我們為什麼需要高級網卡建模，為什麼它對人工智能工作負載尤為重要。

So what's so different here that we need a very detailed modeling.

那麼，這裡有什麼不同之處需要我們進行非常詳細的建模呢？

It's an open source simulator which is already available.

這是一個開放源代碼模擬器，已經可以使用。

I'll explain details of it and how it works along with the RDMA protocol, which is like a major use case in AI workloads.

我將詳細介紹它以及它如何與 RDMA 協議一起工作，這在人工智能工作負載中是一個主要用例。

So then I will actually move to the core of the talk, which is the actual models that we have built for PCA and NIC.

接下來，我將進入演講的核心部分，即我們為 PCA 和 NIC 建立的實際模型。

We'll provide some insights with the data that we have in the current state of the work.

我們將根據目前工作中掌握的數據提供一些見解。

And then we'll call to action on collaboration.

然後，我們將號召大家開展合作。

So if you're also working on building next generation AI infrastructure hardware like us, or really interested in doing a very detailed simulation of these models, do connect with us for that.

是以，如果你也像我們一樣致力於構建下一代人工智能基礎架構硬件，或者真的有興趣對這些模型進行非常詳細的模擬，請與我們聯繫。

So let's start with why this is important work.

讓我們先來看看為什麼這是一項重要的工作。

So in general, simulation is always good.

是以，總的來說，模擬總是好的。

是以，您可以獲得更好的模擬效果。

You will need, you will have better time to market, avoid bugs in the hardware beforehand and all that.

你將需要，你將有更好的時間進入市場，事先避免硬件上的 bug 等等。

So if we take for AI wise, it is more important.

是以，如果我們將人工智能視為明智之舉，它就更為重要。

So everybody knows from morning we are hearing about the scale of the training workloads and the time they run.

是以，每個人都知道，從早上開始，我們就聽到了關於訓練工作量的規模和他們跑步的時間。

So there are other important aspects also apart from scale.

是以，除了規模之外，還有其他重要方面。

So here is like, some of the things are the importance of packet latency and to avoid packet drops here.

是以，這裡有一些事情就像數據包延遲的重要性和避免數據包丟失一樣。

So if you take traditional data center, so there are like many applications talking with many other applications and each of these flows will have different parameters, right?

是以，如果以傳統的數據中心為例，就會有許多應用程序與許多其他應用程序對話，而每個流都會有不同的參數，對嗎？

They will have different timelines, SLAs, some may be, majority of them would be short loop flows.

它們會有不同的時間表和服務水平協議，有些可能是，但大多數都是短循環流程。

Many of them can be also elephant sustained flows.

其中許多還可以是大象持續流。

So the effect of any packet drop here would be actually local.

是以，任何數據包掉落的影響實際上都是局部的。

It will be local to that node, that port, that application.

它將是該節點、該端口、該應用程序的在地數據。

So there will be logic built in the application to actually handle that and it will not affect your entire data center.

是以，應用程序中將內置邏輯來實際處理這一問題，而不會影響整個數據中心。

But that's like very different for AI networks.

但對於人工智能網絡來說，情況就大不一樣了。

So when you start your training job, your packet latency or drop increases, it will affect your entire training time, right?

是以，當您開始培訓工作時，如果數據包延遲或丟包增加，就會影響整個培訓時間，對嗎？

So that's the main thing that why we need to pay close attention at what's happening at NIC and PCI level and try to tune different things or observe different aspects for each of the training loads so that we get better insights.

是以，這就是為什麼我們需要密切關注網卡和 PCI 層發生的事情，並嘗試調整不同的事情或觀察每種訓練負載的不同方面，從而獲得更好的見解的主要原因。

The other aspect is if you see the data transfer on the PCI, it is remarkably different from a normal TCP application.

另一方面，如果你看到 PCI 上的數據傳輸，就會發現它與普通的 TCP 應用程序明顯不同。

So normally the host would actually create, host CPU would create the packets and you would see a lot of PCI writes from the host when it is sending traffic out.

是以，通常情況下，主機會創建數據包，主機 CPU 會創建數據包，當主機向外發送流量時，你會看到主機寫入大量 PCI 數據包。

But for AI workloads, there is a, your RDMA is handled on the NIC, your RNIC.

但對於人工智能工作負載來說，RDMA 是在網卡（即 RNIC）上處理的。

So that is doing the reads from the host memory, right?

這就是從主機內存中讀取數據，對嗎？

So it will also do, update the completion queues.

So the data is like frequent bidirection.

是以，數據就像頻繁的雙向數據。

So the pattern would be like very different.

So if you observe some packet drops or some latency spikes here and there, then it would be really difficult if we don't have insights on what's going on here.

是以，如果您在這裡或那裡觀察到一些數據包掉落或一些延遲峰值，如果我們不能深入瞭解這裡發生了什麼，那將會非常困難。

So that's why we started this work and try to build a very detailed models of this.

是以，我們開始了這項工作，並試圖建立一個非常詳細的模型。

So I'll introduce you to this simulator which we have used.

下面我就向大家介紹一下我們使用過的這個模擬器。

So there are like so many simulators out there.

So there is a simulator to plan your IoT network which are at the edge of the system, right?

是以，有一個模擬器可以規劃處於系統邊緣的物聯網網絡，對嗎？

There's another simulator called Riverbed which Telco operators used to.

還有一種名為 Riverbed 的模擬器，電信運營商經常使用。

So they are like custom built for some purpose.

是以，它們就像是為某種目的而定製的。

Like that, there is another simulator called NS3.

與此類似，還有一款名為 NS3 的模擬器。

你們中的大多數人可能都聽說過。

這是一款非常受歡迎的模擬器。

So that NS3 is a simulator which we all imagine a simulator would be like.

是以，NS3 就是我們想象中的模擬器。

You run it on your normal general purpose system.

你可以在普通的通用系統上運行它。

Every node is virtually represented in some way and you have logical connections between them and you have packet generation and real packets go and you actually do your whole stack processing at every node.

每個節點都以某種方式虛擬表示出來，它們之間存在邏輯連接，可以生成數據包並傳送真正的數據包，而且可以在每個節點上進行整個堆棧處理。

And so it's like a normal simulation but it doesn't work at scale.

是以，它就像一個普通的模擬，但無法大規模運行。

It has like functionally you can do good functionality study like you are developing new protocols, you can use that.

從功能上講，你可以進行很好的功能研究，比如開發新的協議，你就可以使用它。

And some places where you cannot simulate like a medium, like a wireless medium, like how many customers would be there, they would be moving.

有些地方你無法模擬，比如媒介，比如無線媒介，比如那裡會有多少客戶，他們會移動。

So those kind of things like noise, so those kind of things that is very good.

是以，這些類似噪音的東西是非常好的。

But there is this other simulator called HTSim, high throughput simulator.

但還有一種模擬器叫做 HTSim，即高通量模擬器。

It borrows the concepts from NS3 but it's really different.

它借鏡了 NS3 的概念，但確實與眾不同。

So if we see only the performance modeling, we don't really need to process the packet fully, right?

是以，如果我們只看到性能建模，就不需要完全處理數據包，對嗎？

If you see the packet life, it's series of hops and every hop has some latency.

如果你看到數據包的壽命，它是一系列的跳轉，而每一跳都有一定的延遲。

So that latency could be fixed based on whatever you're processing.

是以，延遲時間可以根據處理的內容來確定。

Or it could be like varied a bit based on the size and things like that.

也可以根據尺寸等因素進行調整。

It could be dependent on other things like what other things are happening.

這可能取決於其他因素，比如正在發生的其他事情。

Maybe it's skewed and the scheduler is like picking packets from there.

也許它有偏差，調度程序就像在從中挑選數據包。

So it has dependency on other things happening.

是以，它有賴於其他事情的發生。

So in that sense, if you see, you don't really need real packets also.

是以，從這個意義上說，如果你看到了，你就不需要真正的數據包了。

So in HTSim, there are like no real nodes, just a series of hops, which are logically grouped into some nodes.

是以，在 HTSim 中，好像沒有真正的節點，只有一系列的跳轉，這些跳轉在邏輯上被歸類為一些節點。

You can build whatever you want with that.

你可以用它建造任何你想要的東西。

And you associate different aspects to each of the block so that you can get a simulation model of your entire path.

然後將不同的方面與每個區塊關聯起來，這樣就能得到整個路徑的仿真模型。

So basically, that's a very high level overview.

基本上，這就是一個非常高層次的概述。

There are like some more aspects which I'll cover in the next slide.

我將在下一張幻燈片中介紹其他一些方面。

So this picture basically shows you the actual simulator.

是以，這張圖片基本上展示了實際的模擬器。

It is available for everybody to have a look.

So I'll take an example of a Rocky protocol, which is a major protocol, which is used for RDM applications.

是以，我將以 Rocky 協議為例，這是一個主要協議，用於 RDM 應用。

But the concept equally applies to even TCP, right?

但這一概念同樣適用於 TCP，對嗎？

So basically, you start out doing your simulation and you have your protocol set up and then you want to create a topology.

是以，基本上，您在開始進行仿真時，就已經設置好了協議，然後想要創建一個拓撲結構。

So it supports many topologies, data center topologies.

是以，它支持多種拓撲結構和數據中心拓撲結構。

You can decide how many nodes you want, how many tires you want.

你可以決定要多少節點，多少輪胎。

And then basically, it builds the components in the map, like predetermined fabric completely.

然後，基本上，它會在地圖中構建組件，就像完全預先確定的織物一樣。

And then you send the notion of a packet.

So each of these building blocks will have sinks.

So I'll not go into very much technical details there, but so the sinks form hops and that will actually form a route for each packet and the packets actually follow that path, right?

我就不多說技術細節了，但匯點形成跳數，實際上會為每個數據包形成一條路由，而數據包實際上會沿著這條路徑運行，對嗎？

So here, even though functionality is not needed, at some level, you need to handle the aspects which are related to performance.

是以，即使不需要功能，在某種程度上，也需要處理與性能有關的方面。

Like for example, you need to maintain little state so that you can actually send back the X because your throughput is actually dependent on that.

舉例來說，你需要保持少量狀態，這樣才能真正發回 X，因為你的吞吐量實際上取決於這一點。

So those parts of the protocol you still handle, but you don't have the full stack, right?

是以，協議的這些部分仍由你來處理，但你沒有完整的協議棧，對嗎？

那麼，目前的 HTC 還缺少什麼呢？

So this thing massively scales, but right now, it doesn't have any parts which are related to the PCI or except for basic queuing, it doesn't have anything else on it.

是以，這個東西可以大規模擴展，但現在，它沒有任何與 PCI 相關的部分，或者除了基本的隊列功能外，它沒有任何其他功能。

So that's what we have set up to add, right?

這就是我們要添加的內容，對嗎？

So let's see what kind of data it actually gives, right?

讓我們看看它到底能提供什麼樣的數據吧？