Name: 資料湖倉整合入門指南！ (Intro to Data Lakehouse)
Uploaded: 2025-12-21T12:32:48.000Z
Duration: 5 min 45 s

The History of Data Management in this video, you'll learn about the origin and purpose of the data lakehouse and the challenges of managing big data.

資料管理 在這支影片裡，你會學到 data lakehouse 的起源和目的，以及管理 big data 的挑戰。

To understand what a data lakehouse is, you'll need to explore the history of data management and analytics.

要了解什麼是 data lakehouse，你需要先認識資料管理和分析的歷史。

In the late 1980s, businesses wanted to harness data driven insights for business decisions and innovation.

在 1980 年代後期，企業希望能運用資料驅動的洞察來做商業決策和創新。

To do this, organizations had to move past simple relational databases to systems that could manage and analyze data that was being generated and collected at high volumes and at a faster pace.

為了達成這個目標，組織必須超越單純的關聯式資料庫，轉向能夠管理和分析以高規模和更快速度產生和收集的資料的系統。

Data warehouses were designed to collect and consolidate this influx of data and provide support for overall business intelligence and analytics.

資料倉儲 的設計目的是收集和整合這些湧入的資料，並為整體商業智慧和分析提供支援。

Data in a data warehouse is structured and clean with predefined schemas.

資料倉儲 裡的資料是結構化且乾淨的，並有預先定義好的綱要。

However, data warehouses were not designed with semi structured or unstructured data in mind and became very expensive when trying to store and analyze any data that didn't fit the schema.

然而，資料倉儲 在設計時並未考慮到半結構化或非結構化資料，當試圖儲存和分析任何不符合綱要的資料時，成本變得非常高昂。

As companies grew and the world became more digital, data collection drastically increased in volume, velocity and variety, pushing data warehouses out of favor.

隨著公司成長，世界變得更加數位化，資料收集的規模、速度和多樣性急劇增加，使得資料倉儲 逐漸失寵。

It took too much time to process data and provide results and and there was limited capability to handle data variety and velocity.

處理資料和提供結果耗時太久，而且處理資料多樣性和速度的能力有限。

In the early 2000s, the advent of big data drove the development of data lakes where structured, semi structured and unstructured data could live simultaneously, collected in the volumes and speeds necessary.

在 2000 年代初期，大數據的出現推動了資料湖的發展，讓結構化、半結構化和非結構化資料能夠同時存在，並以必要的規模和速度進行收集。

Multiple data types could be stored side by side in a data lake.

多種資料類型可以並存於資料湖中。

Data created from many different sources such as web logs or sensor data, could be streamed into the data lake quickly and cheaply in low cost cloud object stores.

來自網路日誌或感測器資料等不同來源的資料，可以快速且便宜地以低成本的雲端物件儲存串流到資料湖。

However, while data lakes solved the storage dilemma, it introduced additional concerns and lacked necessary features from data warehouses.

然而，雖然資料湖解決了儲存的難題，但也帶來了額外的擔憂，並缺乏資料倉儲的必要功能。

First, data lakes are not supportive of transactional data and can't enforce data quality, so the reliability of the data stored in the data lake is questionable, mostly due to the various formats.

首先，資料湖不支援交易資料，也無法強制執行資料品質，是以儲存在資料湖中的資料可靠性備受質疑，這主要歸因於其多樣的格式。

Second, with such a large volume of data, the performance of analysis is slower and the timeliness of decision impacting results has never manifested.

其次，由於資料量龐大，分析效能變慢，影響決策的即時結果從未顯現。

And third, governance over the data in a data lake creates challenges with security and privacy enforcement due to the unstructured nature of the contents of a data lake.

再者，由於資料湖內容的非結構化特性，對資料湖中資料的資料治理，在安全性與隱私權執行方面帶來了挑戰。

Because data lakes didn't fully replace data warehouses for reliable BI insights, businesses implemented complex technology stack environments including data lakes, data warehouses, and

因為資料湖無法完全取代資料倉儲以提供可靠的商業智慧洞察，企業實施了複雜的技術堆疊環境，包括資料湖、資料倉儲，以及

additional specialized systems for streaming time series, graph and image databases, to name a few.

用於串流時間序列、圖形和圖像資料庫的額外專門系統。

But such an environment introduced complexity and delay as data teams were stuck in silos completing disjointed work.

但這樣的環境引入了複雜性和延遲，因為資料團隊陷入了孤島，完成零散的工作。

Data had to be copied between the systems and in some cases copied back, impacting oversight and data usage governance, not to mention the cost of storing the same information twice with disjointed systems.

資料必須在系統間複製，有時甚至需要再複製回來，這影響了監督和資料使用資料治理，更不用說重複儲存相同資訊的成本，以及系統之間的斷裂。

Successful AI implementation was difficult and actionable outcomes required data from multiple places.

成功的 AI 實施變得困難，而且可行的成果需要來自多個地方的資料。

資料背後的價值就這樣消失了。

In a recent study By Accenture, only 32% of companies reported measurable value from data.

在最近的一項 Accenture 研究中，只有 32% 的公司回報從資料中獲得了可衡量的價值。

Something needed to change because businesses needed a single flexible, high performance system to support the ever increasing use cases for data exploration, predictive modeling and predictive analytics.

有些東西需要改變，因為企業需要一個單一、彈性、高效能的系統來支援不斷增加的資料探索、預測模型和預測分析的使用案例。

Data teams needed systems to support data applications including SQL analytics, real time analysis, data science and machine learning.

資料團隊需要能夠支援資料應用程式的系統，包括 SQL 分析、即時分析、資料科學和機器學習。

To meet these needs and address the concerns and challenges, a new data management architecture emerged.

為了滿足這些需求並解決疑慮和挑戰，一種新的資料管理架構應運而生。

The Data Lakehouse the Data Lakehouse was developed as an open architecture combining the benefits of a data lake with with the analytical power and controls of a data warehouse built on a data lake.

Data lakehouse 是一種開放式架構，結合了資料湖的優點與資料倉儲的分析能力和控制功能，並建構在資料湖之上。

A data lakehouse can store all data of any type together, becoming a single reliable source of truth providing direct access for AI and bi.

data lakehouse 可以儲存任何類型的所有資料，成為單一可靠的真相來源，為 AI 和 BI 提供直接存取。

Together, Data lakehouses like the Databricks Lakehouse platform offer several key features such as transaction support including ACID transactions for concurrent read write interactions, schema enforcement and governance for data integrity and robust auditing needs.

總體而言，像 Databricks Lakehouse 平臺這樣的 data lakehouse 提供了幾項關鍵功能，例如支援交易，包括 ACID 交易以實現並行讀寫互動、綱要強制執行和資料治理以確保資料完整性，以及強大的稽核需求。

Data governance to support privacy regulation and data use metrics BI support to reduce the latency between obtaining data and drawing insights.

資料治理以支援隱私權法規和資料使用指標，商業智慧支援以縮短取得資料到獲得洞察的時間。

Additionally, the Data Lakehouse offers decoupled storage from compute, meaning each operates on their own clusters allowing them to scale independently to support specific needs.

此外，Data Lakehouse 提供運算與儲存分離，這表示它們各自在自己的叢集上運作，能獨立擴展以支援特定需求。

Open storage formats such as Apache Parquet, which are open and standardized so a variety of tools and engines can access the data directly and efficiently Support for diverse data types so a business can store, refine, analyze and access semi structured, structured and unstructured data in one location.

開放式儲存格式，如 Apache Parquet，因為開放且標準化，所以各種工具和引擎都能直接有效率地存取資料；支援多樣化的資料類型，讓企業能在一個地方儲存、精煉、分析和存取半結構化、結構化和非結構化資料。

Support for diverse workloads Allowing a range of workloads such as data science, machine learning and SQL analytics to use the same data repository and end to end streaming for real time reports removes the need for a separate system dedicated to real time data applications.

支援多樣化的工作負載，讓資料科學、機器學習和 SQL 分析等各種工作負載都能使用相同的資料儲存庫，而端對端串流用於即時報表，則無需另外建置專門處理即時資料應用程式的系統。

The Lakehouse supports the work of data analysts, data engineers and data scientists all in one location.

Lakehouse 在單一地點支援資料分析師、資料工程師和資料科學家的工作。

The Lakehouse essentially is the modernized version of a data warehouse, providing all the benefits and features without compromising the flexibility and depth of a data lake.

基本上，Lakehouse 就是資料倉儲的現代化版本，提供所有優點和功能，同時不犧牲資料湖的彈性和深度。

字幕列表影片播放

資料湖倉整合入門指南！ (Intro to Data Lakehouse)

access

multiple

compromise

simultaneously