Name: 反思我們如何為人工智能代理搭建腳手架 - Rahul Sengottuvelu，Ramp (Rethinking how we Scaffold AI Agents - Rahul Sengottuvelu, Ramp)
Uploaded: 2025-03-22T20:02:21.000Z
Duration: 16 min 32 s

I've been working on LLMs for four years, which is, well, which is a kind of a long time, I guess, in LM land.

我攻讀法學碩士已經四年了，我想，在法學碩士領域，這已經是一段很長的時間了。

Everything started happening really when ChatGPD came out.

ChatGPD 問世後，一切才真正開始。

So, I was trying to build what people would now call an AI agent company.

是以，我試圖建立一家人們現在稱之為人工智能代理公司的公司。

Back then, we were just doing customer support.

We're trying to make our chatbot smarter, and we're trying to figure out what models to use, or what tech to use to get them to respond to customers better.

我們正試圖讓我們的哈拉機器人變得更聰明，我們正試圖找出該使用什麼模型，或使用什麼技術來讓它們更好地迴應客戶。

And we were messing with GPD 2 on support, and models were so frustratingly stupid, and the context windows were small, and they were not very smart reasoning, and it was just incredibly annoying.

我們在支持系統上使用 GPD 2，模型愚蠢得令人沮喪，上下文窗口很小，推理也不太智能，簡直令人難以置信的煩人。

And we just wrote lots of code around these models to get them to work, at least somewhat reliably.

我們只是圍繞這些模型編寫了大量代碼，讓它們至少在一定程度上可靠地工作。

And along the way, as models got smarter, this kind of had to delete more of that code.

一路走來，隨著模型越來越智能，我們不得不刪除更多的代碼。

And this ended up seeing a lot of patterns in what code needs to get deleted, how to build agents in what ways that will scale with more intelligence.

最終，我們發現了很多模式，比如哪些代碼需要刪除，如何以何種方式構建代理，從而隨著智能的提高而擴展。

And clearly, we're going to continue to get a lot more intelligence.

顯然，我們將繼續獲得更多情報。

And I just wanted to maybe talk about a single idea throughout the talk through various examples.

我只是想通過各種例子，在整個講座中談談一個想法。

We'll do some setting, but I'll also have a bunch of demos to kind of drive home the point, and maybe I can convince you guys that there's a certain way of building agents that's slightly better than other ways.

我們會做一些設置，但我也會做一些演示，讓大家明白這一點，也許我能讓你們相信，有一種建立代理的方法比其他方法略勝一籌。

I also built a structure extraction library called JSONformer.

我還創建了一個名為 JSONformer 的結構提取庫。

I'm not fully sure, but timing-wise, it was before all the other major ones.

我不太確定，但從時間上看，是在所有其他重大事件之前。

And that was also scaffolding around a model.

這也是圍繞模型搭建的腳手架。

Models were too stupid to output JSON, and we were just really begging and pleading it, and forcing it to act in ways that we want it to be.

模型太笨，無法輸出 JSON 格式，我們只能苦苦哀求，迫使它按照我們的意願行事。

So as I said earlier, I just have one core agenda item here, which is I want to convey one idea.

是以，正如我剛才所說，我在這裡只有一個核心議程項目，那就是我想傳達一個想法。

We'll start off, all of you probably read the essay a bit or less, and just quickly go through what it is.

我們先來看看這篇文章，你們可能都或多或少地讀過一點，快速瀏覽一下它的內容。

We'll go through a production agent we have at Ramp, and how it works, and three different ways of architecting it.

我們將介紹 Ramp 的一個生產代理，以及它的工作原理和三種不同的架構方法。

And then I have a demo to really push maybe how we all think about how software and backends and things will work in the future.

然後，我有一個演示，也許能真正推動大家思考軟件和後端以及其他東西在未來的工作方式。

So very simply, the idea is just that systems that scale to compute beat systems that don't.

是以，很簡單，我們的想法就是，能夠擴展計算能力的系統會擊敗不能擴展計算能力的系統。

So there's two systems, and without any effort, one of the systems can just think more or use more compute in some way.

是以，有兩個系統，不費吹灰之力，其中一個系統就能以某種方式進行更多思考或使用更多計算。

That system tends to beat systems that are rigid and fixed and just deterministic.

這種系統往往會擊敗那些僵化、固定和只是確定性的系統。

So from that idea, it's pretty clear, like, if you're building systems, you might as well build systems that improve with more compute.

是以，從這一理念出發，很明顯，如果你要構建系統，你不妨構建能隨著計算量增加而改進的系統。

And this seems pretty obvious, like, obvious conclusion from the bitter lesson.

這似乎很明顯，就像從慘痛教訓中得出的顯而易見的結論。

Taking it a step further, why is this true?

再進一步說，為什麼會這樣呢？

Like, they just don't exist, and most things in the world aren't exponential.

比如，它們根本就不存在，世界上大多數事物都不是指數型的。

So when you find one, you just should hop on, strap on, and just take the free pass and go for the ride.

是以，當你找到一輛車時，就應該跳上去，繫上安全帶，拿著免費通行證去兜風。

And there's a lot of examples from history that kind of reflect this.

歷史上有很多例子都反映了這一點。

So for chess and Go and computer vision, Atari games, like, people have tried to build lots of systems and written a lot of code.

是以，對於國際象棋、圍棋、計算機視覺、雅達利遊戲等，人們已經嘗試構建了很多系統，編寫了大量代碼。

And my way of thinking about rigid systems is just, like, spending a lot of time writing weekends and writing very clever software, well-abstracted, maybe trying to synthesize human reasoning and thought process into features and then using them in clever ways and trying to approximate how a human would think.

我對僵化系統的思考方式就是，花大量時間在週末寫作，編寫非常聰明的軟件，抽象得很好，也許是想把人類的推理和思維過程綜合到功能中，然後以聰明的方式使用它們，並試圖接近人類的思維方式。

And if you actually fix the amount of compute, that approach will win.

如果計算量確實是固定的，這種方法就會獲勝。

But if it just turns out, if you end up scaling up how much search you're doing, the general method always ends up winning, even, like, in all these cases, so Atari, Go, and computer vision.

但如果事實證明，如果你最終擴大了搜索量，一般方法最終總會獲勝，甚至在所有這些情況下，比如雅達利、圍棋和計算機視覺。

So RAMP is a finance platform that helps businesses manage expenses, payments, procurement, travel, bookkeeping more efficiently.

是以，RAMP 是一個財務平臺，可幫助企業更高效地管理支出、付款、採購、差旅和簿記。

And we have a ton of AI across the product, so automate a lot of boring stuff the finance teams do and employees do with submitting expense reports and booking your flights and hotels and submitting reimbursements, all of that.

我們在整個產品中採用了大量人工智能技術，是以可以自動處理財務團隊和員工在提交費用報告、預訂機票和酒店、提交報帳單等方面所做的大量枯燥工作。

And so a lot of the work behind the scenes is just we're interacting with other systems and helping, like, legacy systems and helping employees get their work done faster.

是以，很多幕後工作都是我們與其他系統交互，幫助傳統系統，幫助員工更快地完成工作。

So let's actually talk through one of the systems we have today at RAMP, and maybe some talk through, like, the different versions of the system and how it developed over time.

是以，讓我們實際談談 RAMP 目前擁有的一個系統，或許還可以談談該系統的不同版本，以及它是如何隨著時間的推移而發展的。

So we're going to talk about something called a switching report.

是以，我們要討論一種叫做切換報告的東西。

All it needs to do is take in a CSV, a CSV in arbitrary format, so the schema could be seriously anything from the internet.

它只需要接收 CSV，一個任意格式的 CSV，所以模式可以是互聯網上的任何東西。

And we want these CSVs to come from third-party card providers.

我們希望這些 CSV 來自第三方卡提供商。

So when people onboard to RAMP, we want to give them a nice checklist and say, hey, here are all the transactions you have on other platforms, and we want to help you move them over.

是以，當人們加入 RAMP 時，我們希望給他們一個很好的清單，並說，嘿，這是你在其他平臺上的所有交易，我們希望幫助你將它們轉移到其他平臺上。

And the more transactions come on RAMP, the more we can help you, and the more you'll use our software, and the more everyone benefits.

在 RAMP 上進行的交易越多，我們就能為您提供越多的幫助，您就會越多地使用我們的軟件，每個人都會從中受益。

And so the switching report is just really a checklist.

是以，切換報告實際上只是一份核對表。

But to read people's CSV transactions, we need to understand those, and other platforms have all these kinds of crazy schemas.

但要讀取人們的 CSV 交易，我們需要理解這些內容，而其他平臺有各種瘋狂的模式。

And so the description of the problem we have here is just for an arbitrary, arbitrary, like, CSV, how can we support parsing it and then into some format that we understand?

是以，我們所面臨的問題是，對於任意的 CSV 文件，我們如何才能支持對其進行解析，然後將其轉換成我們能夠理解的格式？

So let's just start with the simple approach, right?

所以，讓我們從簡單的方法開始，對嗎？

It's like, let's just take the 50 most common third-party card vendors, and let's just manually write code for all of them.

這就好比，讓我們把 50 家最常見的第三方卡供應商都找出來，然後為所有這些供應商手動編寫代碼。

Now, obviously, like, this will just work.

現在，很明顯，就像，這將只是工作。

It is some work, not a lot of work, but you still have to maybe go to 50 different platforms and download their CSVs, see what schemas they have, and then write code.

雖然工作量不大，但你還是得去 50 個不同的平臺下載 CSV，看看它們有哪些模式，然後編寫代碼。

Maybe if they decide one day they change their format, your thing will break, but that's okay, you'll get paged, and you can wake up and go fix it.

也許如果有一天他們決定改變格式，你的東西就會壞掉，但沒關係，你會收到傳呼，醒來後就可以去修理。

So let's maybe introduce some LLMs in here.

所以，讓我們在這裡介紹一些法學碩士吧。

So from the over-engineered code where you ended up writing 100,000 lines, maybe we want a more general system.

是以，從你最終要寫 10 萬行的過度工程代碼來看，也許我們需要一個更通用的系統。

So let's introduce a little bit of LLMs, a little bit of AI in here.

是以，讓我們在這裡介紹一下法律碩士和人工智能。

And so in the deterministic flow, let's maybe add some, or just like scripting in classical scripting land, let's add some more calls to OpenAI, or you have an embedding model, you want to do somatic similarity or something like that.

是以，在確定性流程中，我們或許可以添加一些，或者就像在經典腳本領域編寫腳本一樣，我們可以向 OpenAI 添加更多調用，或者你有一個嵌入模型，你想做體細胞相似性或類似的事情。

So then let's just take every column in the CSV that comes in.

是以，我們只需獲取 CSV 中的每一列。

Let's try to classify what kind of column it is.

讓我們試著把它歸類為哪一類欄目。

And then we map it on, and then we probably could end up in a schema that we're happy with.

然後我們將其映射上去，最後可能會得到一個我們滿意的模式。

Again, most of the compute is running in classical land.

同樣，大部分計算都是在經典環境中運行的。

Some of it is running in fuzzy LLM land, but this is somewhat looking like a more general system.

其中有些是在模糊的法律碩士領域運行的，但這有點像一個更普遍的系統。

Let's go maybe a different approach, and like we just go all the way through.

也許我們可以換一種方式，就像一路走過去。

Let's just say we're just going to literally give the CSV to an LLM and say, you have a code interpreter, so you can write whatever code you want, pandas or all the faster Rust-based ones.

比方說，我們把 CSV 交給一個 LLM，然後說，你有一個代碼解釋器，所以你可以寫任何你想寫的代碼，pandas 或者所有更快的基於 Rust 的代碼。

你有所有這些 Python 套裝軟體。

You're allowed to look at the head of the CSV, the tail, whichever rows you want.

你可以查看 CSV 的頭部和尾部，無論你想查看哪一行。

And then I just want you to give me a CSV with this specific format.

然後，我只想讓你給我一個特定格式的 CSV。

Here's a verifier that you can use to tell if it's working or not.

這裡有一個驗證器，您可以用它來判斷是否有效。

Turns out this approach actually doesn't work.

事實證明，這種方法實際上行不通。

But instead, if you run it 50 times in parallel, it's actually very likely that it works really well and generalizes across a ton of different formats.

但相反，如果並行運行 50 次，實際上很有可能效果非常好，並能在大量不同的格式中通用。

The amount of compute here is actually probably like, what is that number?

10,000 times more than the first approach we came up with.

比我們想出的第一種方法多出一萬倍。

But again, like, what is truly scarce in the world is engineer time.

但還是那句話，世界上真正稀缺的是工程師的時間。

Maybe not in a while, but at least today.

也許暫時不會，但至少今天會。

And we'd rather have a system that works really well, and even with the 10,000 times more compute, it will probably cost less than a dollar, and every transaction that's switched over, every failed CSV will cost RAMP way more money than whatever money we spend on this exact architecture.

我們寧可要一個運行得非常好的系統，即使計算量增加一萬倍，它的成本也可能不到一美元，而每一筆交易的切換、每一個失敗的 CSV，都會讓 RAMP 損失比我們花在這個架構上的錢更多的錢。

是以，這是一個非常具體的例子。

It's like, how does this apply to the agents that we all build, and maybe the systems we're all working on?

這就好比，這如何適用於我們構建的代理，也許是我們正在開發的系統？

Turns out, something like this actually generalizes.

事實證明，這樣的事情其實是可以推廣的。

So if you look at the three approaches, and let's assume, like, the black arrow is just classical compute, and then the blue arrows are fuzzy land.

所以，如果你看看這三種方法，假設黑色箭頭是經典計算，藍色箭頭是模糊計算。

So it goes into neural net, and all sort of weird matrix multiplication happens, and then we're in latent space, and it gets all alien intelligency, and then comes back to classical land.

於是它進入神經網絡，發生各種奇怪的矩陣乘法，然後我們就進入了潛空間，它獲得了所有外星智能，然後又回到了經典領域。

We just wrote code, and it just worked, mostly.

我們只需編寫代碼，大部分時間都能正常工作。

The constrained agent, so the second approach, we broke into fuzzy land from classical land when we decided we wanted similarity scores or something like that.

受限代理，所以第二種方法，當我們決定需要相似性分數或類似的東西時，我們從經典領域進入了模糊領域。

And then the third approach is actually flipped, where the LLM decides it needs to go into classical land.

第三種方法實際上是翻轉的，即法律碩士決定需要進入古典領域。

So it writes some code, it writes some pandas or Python code, and it decides to break in into this classical land when it needs to, but most of the compute is fuzzy.

是以，它編寫了一些代碼，編寫了一些 pandas 或 Python 代碼，並決定在需要時進入這個經典領域，但大部分計算都是模糊的。

Actually, this is maybe not the most accurate graph, like, because it proposed that we run it 50 times.

實際上，這可能不是最準確的圖表，因為它建議我們運行 50 次。

But if you look at a back end in general, they're all request response.

但如果你看一下一般的後端，它們都是請求響應。