Name: 大數據與快速數據：優化人工智能戰略 (Big Data vs Fast Data: Optimize Your AI Strategy)
Uploaded: 2025-06-05T03:59:44.000Z
Duration: 15 min 27 s

Data is the foundation of AI and automation, but not all data is the same.

數據是人工智能和自動化的基礎，但並非所有數據都一樣。

情態助動詞 B1

And if you don't understand the difference between big data and fast data, you might be building your AI strategy on the wrong foundation.

如果您不瞭解大數據和快速數據之間的區別，您可能會在錯誤的基礎上制定人工智能戰略。

轉承詞

We've talked about big data a lot, and there are plenty of systems that are optimized for big data, however, they might not be best suited for fast data.

我們經常談論大數據，有很多系統都針對大數據進行了優化，但它們可能並不最適合快速數據。

This is a real problem for technologists today, for when we're working with different kinds of data, we need to make sure we're putting it on the right architecture.

這是當今技術人員面臨的一個真正問題，因為當我們處理不同類型的數據時，我們需要確保將其放在正確的架構上。

So do you think you optimize for scale and deep insights, or more so for real-time responsiveness?

那麼，您認為自己是為了規模和深入洞察而進行優化，還是為了實時響應而進行優化？

Today we're going to break down how to make the right choices here.

今天我們就來分析一下如何在這裡做出正確的選擇。

And let's go through exactly what these two categories mean, why they actually represent a trade-off, and how understanding where you fit will directly impact your ability to scale.

讓我們來了解一下這兩個類別的確切含義，為什麼它們實際上代表著一種權衡，以及瞭解自己的定位將如何直接影響自己的擴展能力。

So, while we're going to go through two definitions here, I want you to really think about how this not only are two different categories, but essentially represent a trade-off.

是以，當我們在這裡討論兩個定義時，我希望你們真正思考一下，這不僅是兩個不同的類別，而且本質上代表了一種取捨。

Because as you start to optimize for big data, you lose the flexibility to gain value out of fast data.

因為當你開始優化大數據時，你就失去了從快速數據中獲取價值的靈活性。

So you need to make sure that you're always comparing these two to really figure out which category your work really falls into.

是以，你需要確保經常對這兩者進行比較，以真正弄清你的作品到底屬於哪一類。

And it's all about where we get value from data.

而這一切都與我們從數據中獲取價值有關。

Are we getting value from fast data, or from the fact that there is a lot of data to gain insights from?

我們是從快速數據中獲得價值，還是從大量數據中獲得洞察力？

So there isn't really a silver bullet technology.

是以，並不存在真正的靈丹妙藥技術。

These are two completely different kinds of architectures and technology suites.

這是兩種完全不同的架構和技術套件。

So you really have to make sure that you pick a side, basically, and you're going to optimize for that.

是以，你必須確保你基本上選擇了一方，併為此進行優化。

Maybe you'll use them in combination, but let's start by going through the definition of what these really mean.

也許你會把它們結合起來使用，但讓我們先來了解一下它們的真正含義。

So, first things first, we'll talk about big data.

所以，首先我們來談談大數據。

You probably work with big data all the time.

We've probably been talking about it for about over a decade now.

我們大概已經討論了十多年了。

And this is basically where we're trying to analyze massive amounts of data sets to extract insights over time.

基本上，這就是我們試圖分析海量數據集，以便隨著時間的推移提取洞察力的地方。

If your goal is to train AI models, analyze historic patterns, or manage massive data archives, you're going to be dealing with some kind of big data.

如果您的目標是訓練人工智能模型、分析歷史模式或管理海量數據檔案，您就需要處理某種大數據。

And you're going to see a bigger focus on big data when you have really important compliance and governance requirements.

當你有非常重要的合規性和治理要求時，你會發現大數據會受到更多關注。

So, when you're building a big data architecture, you're going to see some common themes.

是以，在構建大數據架構時，你會看到一些共同的主題。

And really the biggest and most important thing about big data is the data storage and management component.

實際上，大數據最大、最重要的是數據存儲和管理部分。

So, that really relies on your data warehouse, which is going to be some kind of very large data repository where all this data is stored.

是以，這實際上依賴於你的數據倉庫，它將是某種非常大的數據存儲庫，所有這些數據都存儲在這裡。

And that's where you're getting your value, right?

The fact that you can put lots and lots of information, more than you ever could before, in one place to build value from it.

事實上，您可以將大量的資訊（比以往任何時候都多）集中到一個地方，並從中創造價值。

And then what really comes with this as another key technology would be something to help process and manipulate the data in some way.

作為另一項關鍵技術，它還能幫助處理和操作數據。

So, you can then extract even more value out of your data.

是以，您可以從數據中提取更多價值。

So, this could be some kind of automation or processing technology like Spark, for example.

是以，這可能是某種自動化或處理技術，例如 Spark。

And that's essential when you're working with big data.

The other piece of this you're going to see would be business insights and AI platforms.

你將看到的另一塊內容是業務洞察和人工智能平臺。

So, you're going to want to create dashboards from this data.

是以，你需要根據這些數據創建儀表盤。

You're going to want to create different kinds of models.

You're going to want to get more insights from how we're actually using this data.

你會希望從我們如何實際使用這些數據中獲得更多洞察力。

So, in that, you're going to see lots of technologies around data visualization and AI platforms so data scientists can actually work with the data.

是以，你將會看到大量圍繞數據可視化和人工智能平臺的技術，這樣數據科學家就能真正使用數據工作。

So, these are really the core technologies you're going to be seeing when talking about big data and the kind of architecture you're going to be driving towards.

是以，在談論大數據和架構時，這些才是真正的核心技術。

If you're working on AI model training or any kind of predictive analysis or even think about deep learning, these are the kind of investments that make sense because they're going to help you scale in depth.

如果你正在進行人工智能模型訓練或任何類型的預測分析，甚至考慮深度學習，這些投資都是有意義的，因為它們將幫助你進行深度擴展。

現在，讓我們來談談快速數據。

And to no surprise, fast data is all about speed, whereas big data was all about depth.

毫無疑問，快速數據講究的是速度，而大數據講究的是深度。

With fast data, it really is more about how we can make instant decision making.

有了快速數據，我們就能真正做到即時決策。

And this could be for fraud detection, personalization, or some kind of Internet of Things automation, just as some examples.

例如，這可以用於欺詐檢測、個性化或某種物聯網自動化。

This is not to say that fast data can't be large.

這並不是說快速數據不可能很大。

It really has to do with where does the value come from?

Is the data valuable at that point in time or is it valuable in aggregation over a long period of time?

數據是在當時有價值，還是在長期彙總後有價值？

And then based on the answer to that question, you can start to really see the differences here.

然後根據這個問題的答案，你就可以開始真正看到這裡的不同之處了。

And so with big data, if you wanted to forecast your sales for next year using past and historic sales as evidence, that would be a big data use case.

是以，利用大數據，如果您想以過去和歷史銷售額為依據預測明年的銷售額，這就是大數據用例。

If you want to know what your sales were in the last five minutes, that would be fast data.

如果您想知道過去五分鐘的銷售情況，這就是快速數據。

And you might use that to make decisions going forward.

你可以利用這一點來做出今後的決定。

So both are incredibly important and powerful, but it really has to do with the difference in where that data value lies.

是以，兩者都無比重要和強大，但真正的區別在於數據價值的不同。

Now you're going to see different kinds of investments as well.

現在，你還會看到不同種類的投資。

With fast data, you're really going to see more investments in data integration.

有了快速數據，你就會真正看到對數據集成的更多投資。

So definitely for pretty much all fast data, the cornerstone of this really is going to be some kind of streaming or something like Kafka.

是以，對於幾乎所有的快速數據來說，其基石都將是某種流或類似 Kafka 的東西。

That's going to take all these little data events and is going to aggregate them and send them off to another system so that we can actually then bring them in and take some kind of action on them.

它將接收所有這些小數據事件，並將其彙總，然後發送到另一個系統，這樣我們就可以將其導入系統，並對其採取某種行動。

So this would lead to some kind of system that would actually do some kind of event, right?

是以，這將導致某種系統能夠真正完成某種活動，對嗎？

You want to take that data and then trigger something off of it.

你要獲取這些數據，然後從中觸發一些東西。

So you have your stream here, which is an incredibly important piece of technology.

是以，這裡有你的數據流，這是一項非常重要的技術。

And then this is probably going to link to some kind of function as a service or some kind of very low latency, lightweight processing structure where you can trigger and run this event.

然後，這很可能會鏈接到某種功能即服務，或者某種延遲極低的輕量級處理結構，在這種結構中，你可以觸發並運行該事件。

And this is basically going to allow us to make very quick decisions that are really just siloed and isolated and completely independent from each other and do not run as an aggregate system per se.

從根本上說，這將使我們能夠做出非常快速的決策，而這些決策實際上只是各自為政、相互孤立、完全獨立，本身並不作為一個整體系統運行。

And then the last piece of this really is a little bit of storage.

Usually storage that's ephemeral, maybe lives on the edge or could be a cache so that you could actually take a couple of these events and a couple of these data points that we all agree are very important and start bringing them all in to store them in the short term when they have value.

通常情況下，存儲是短暫的，可能存在於邊緣，也可能是一個緩存，這樣你就可以實際獲取一些我們都認為非常重要的事件和數據點，並在它們具有價值時，開始將它們全部引入並在短期內存儲起來。

So if you want to know what happened in the last five minutes, what happened in the last hour, you can kind of keep an inventory of this.

是以，如果你想知道過去五分鐘內發生了什麼，過去一小時內發生了什麼，你就可以記錄下這些資訊。

It's not the last place that it's going to be stored or its permanent data warehouse.

它不是最後的存儲地點，也不是永久的數據倉庫。

But it's needed to facilitate that value of when you need data in real time.

但是，當您需要實時數據時，需要它來促進這種價值。

So these are really our core key differences here in architectures.

是以，這些就是我們在架構方面的核心關鍵差異。

But let's go ahead and take a look now at what kind of maturity models we really have around both of these kinds of data.

不過，現在讓我們來看看圍繞這兩種數據，我們到底有什麼樣的成熟模型。

So now that we understand what big data is and the kind of architecture it generally has, let's get an understanding of different levels of maturity when working with big data.

既然我們已經瞭解了什麼是大數據以及它一般具有的架構類型，那麼讓我們來了解一下使用大數據時的不同成熟度。

So let's break this down into crawl, walk, run, or kind of like beginner, intermediate, advanced models of what it really looks like to work with big data.

是以，讓我們把它抽成爬、走、跑，或者初級、中級、高級模式，看看使用大數據到底是什麼樣子。

Most of us have started in this stage where we have many different data silos in our organization.

我們大多數人都是從這個階段開始的，在我們的組織中存在許多不同的數據孤島。

And within those silos, you know, we have our data repositories, which is great.

在這些孤島中，我們有自己的數據存儲庫，這很好。

Maybe we're doing a bit of AI off of some.

也許我們正在利用一些人工智能。

Maybe we're building some dashboards off of others.

也許我們在別人的基礎上建立了一些儀表盤。

And, you know, we're generating business value.

而且，你知道，我們正在創造商業價值。

我們正在從數據中發現新的東西。

So there's really nothing wrong with this kind of architecture.

是以，這樣的建築結構並沒有什麼問題。

This is naturally where most people start out when working with some kind of big data architecture.

這自然是大多數人在使用某種大數據架構時的出發點。

And then quickly, you'll start to realize that you find more optimization by generally bringing everything together.

然後很快，你就會開始意識到，一般來說，把所有東西集中在一起，你就能找到更多的優化方法。

So moving all these sources to one larger data repository where they can reside either as like a data fabric or a data mesh or literally storing them all in the same kind of location.

是以，將所有這些數據源轉移到一個更大的數據存儲庫中，在那裡，它們可以像數據結構或數據網格一樣駐留，或者將它們全部存儲在同一個位置。

And this is where you'll start to see the introduction of some kind of processing technology.

這時，你就會看到某種加工技術的引入。

So you're starting to work with big data technologies.

所以你們開始使用大數據技術。

You're starting to really see the different kind of connections that can be formed.

你開始真正看到可以建立的各種聯繫。

You're finding those basically economies of scale of bringing new things together.

你會發現，將新事物整合在一起，基本上可以實現規模經濟。

It's kind of the natural progression from that stage.

Now, taking this even farther, because I think many people have data warehouses.

現在，再進一步說，我想很多人都有數據倉庫。

It's really about adding actually AI and automation to this kind of architecture.

這實際上是在這種架構中加入人工智能和自動化。

So in your data repository, we would really expect to have some kind of auto scaling at the storage level.

是以，在您的數據存儲庫中，我們確實希望在存儲層面上實現某種自動擴展。

So we could actually work with different levels of storage that change dynamically based on business need.

是以，我們實際上可以根據業務需求動態改變不同級別的存儲。

As well as we would also want this just totally encompassed in some kind of smart or auto governance kind of structure so that you have this kind of locked down.

此外，我們還希望將其完全納入某種智能或自動治理結構中，這樣你就可以將其鎖定。

實際上，這可以由人工智能來驅動。

So you can actually be enhancing your data architecture in general with AI.

是以，您實際上可以利用人工智能來增強您的數據架構。

And that's really what's going to take you to a more advanced place where you can load more data.

這將帶你進入一個更高級的地方，在那裡你可以加載更多的數據。

And a lot of the basically the maintenance that comes with this can be automated.

很多維護工作基本上都可以自動完成。

And you can move faster and focus on the actual business insights and AI models that can be generated out of this data.

您可以更快地專注於實際業務洞察力和人工智能模型，這些都可以從這些數據中生成。

As opposed to focusing too much on maintenance and organizational silos.

而不是過分關注維護和組織孤島。

So for example, an organization might start from siloed data warehouses to a unified data system or fabric.

是以，舉例來說，一個組織可能會從孤立的數據倉庫轉向統一的數據系統或結構。

And then further enhance their big data architecture with AI and automation to make it as optimized as possible.

然後利用人工智能和自動化進一步增強大數據架構，使其儘可能優化。

字幕列表影片播放

大數據與快速數據：優化人工智能戰略 (Big Data vs Fast Data: Optimize Your AI Strategy)

ultimately

incredibly

massive

process