Name: Unity Catalog 總覽！ (Unity Catalog Overview)
Uploaded: 2025-12-21T12:43:16.000Z
Duration: 6 min 48 s

情態副詞

Let's get a look at some of those features matej has highlighted that we've recently added to Unity Catalog.

讓我們來看看 matej 提到的、我們最近為 Unity Catalog 新增的一些功能。

We'll go ahead and start in Data Explorer where we can see all of the objects registered to our Unity catalog.

我們就從 Data Explorer 開始，在這裡可以看到註冊到我們 Unity Catalog 的所有物件。

To get us started, I'll open up Retail Prod where I know our team is building a Churn prediction model.

為了開始，我會打開 Retail Prod，我知道我們的團隊正在那裡建立一個 Churn prediction model。

What I love about this view is that I can see that Churn prediction model along with previous versions metadata comments right alongside the data that was used to train it.

我喜歡這個檢視的地方是，我可以同時看到這個 Churn prediction model、先前版本的 metadata comments，以及用來訓練它的資料。

I can look at sample data history and lineage and I can view non tabular data or files and functions associated with the project as long as they're registered to the same schema.

我可以查看範例資料歷史記錄和 lineage，並且可以檢視與該專案相關的非 tabular data 或檔案和 functions，只要它們已註冊到相同的 schema。

You might also notice that with Lakehouse Federation you can now access and external Data systems like MySQL, Postgres, and

你可能也會注意到，透過 Lakehouse Federation，你現在可以直接從 Data Explorer 存取外部 Data systems，像是 MySQL、Postgres 和

Snowflake from directly within Data Explorer.

This is huge because lots of organizations still have a long tail of sources yet to be ingested into the lakehouse, so you want to be able to permission manage and access that from one place.

這非常重要，因為許多組織仍然有許多尚未 ingest 到 lakehouse 的來源，所以你希望能從一個地方進行 permission manage 和存取。

For example, if I want to give my analyst team read access to query this table, all I have to do is click Permissions, highlight that group, hit select and boom.

例如，如果我想讓我的 analyst 團隊擁有查詢這個 table 的 read 權限，我只需要點擊 Permissions，選取該群組，然後點擊 select，就這麼簡單。

Now they can query the table with all of the bells and whistles available to Unity Catalog like masking and tagging and in a future version this will push down all the way into the source system itself, in this case postgres.

現在他們可以用 Unity Catalog 提供的所有進階功能來查詢這個 table，像是 masking 和 tagging，未來版本中，這將會直接推送到來源系統，在這個例子中是 postgres。

Data Explorer is great when I know which specific assets I want to work with and I know specifically what I want to do.

當我知道我想要處理的特定 assets 以及我想要做的具體事情時，Data Explorer 是個很棒的工具。

But if what I am really looking to get is a high level understanding of the health of my data estate, I want to open up this Unity Catalog governance portal.

但如果我真正想做的是對我的 data estate 的整體健康狀況有一個高層次的瞭解，我會打開這個 Unity Catalog governance portal。

This will give me a sense of what all is inside my data ecosystem and how we're doing against governance and monitoring against them.

這將讓我瞭解我的 data ecosystem 中有哪些東西，以及我們在 governance 和 monitoring 方面的表現如何。

For example, of my 143,000 tables I can see that about 60% of them are monitored.

舉例來說，在我擁有的 143,000 個 table 中，大約有 60% 受到監控。

They have Lakehouse Monitoring enabled which is AI for governance.

它們啟用了 Lakehouse Monitoring，這就是 AI for governance。

With Lakehouse Monitoring you can set up alerts and dashboards to automatically detect and automatically correct lots of issues with respect to potential PII leakage, data quality, data drift and more.

透過 Lakehouse Monitoring，你可以設定警報和儀錶板，自動偵測並自動修正許多潛在的 PII 洩漏、資料品質、資料漂移等問題。

So this dashboard gives us a view of how our data estate changes over time, how we're doing on our governance and monitoring coverage, which of my catalogs might need attention based on some high level health metrics and one of the features I'm most excited about this surfaces some of those Auto detected issues for your review.

所以這個儀錶板讓我們能夠瞭解我們的 data estate 隨時間的變化，我們在 governance 和 monitoring coverage 上的表現，哪些 catalog 可能需要關注，以及我最期待的功能之一，它會顯示一些自動偵測到的問題供你審查。

Now, if you remember, Retail Prod is my team's project, so I'm definitely concerned that there's some potential personally identifiable information leaking into this table.

如果你還記得的話，Retail Prod 是我們團隊的專案，所以我當然很擔心這個 table 可能會洩漏個人身份資訊。

Let's go ahead and open up the lakehouse monitoring dashboard for this table to get a better sense of what's going on.

我們就來打開這個 table 的 lakehouse monitoring dashboard，更清楚地瞭解目前的情況。

What's special about this view, where we can see this one active alert, is that we can actually see how this table has evolved over time.

這個檢視的特別之處，也就是我們可以看到這個活躍的警報，是我們可以實際看到這個 table 隨著時間的演變。

I can also see that that alert only started as of today, which is great.

我也可以看到這個警報是從今天才開始的，這很棒。

Hopefully that gives us enough time to contain anything that might have been impacted.

希望這樣我們就有足夠的時間來處理任何可能受影響的狀況。

So if we scroll down, we can get a bit more information about what the system saw.

所以如果我們往下滑動，我們可以獲得更多關於系統偵測到的資訊。

Like this PII appeared in user attributes.

像是 PII 出現在 user attributes 中。

It looks like 7.8% of the rows may have a first or last name in that column.

看起來有 7.8% 的資料列，在這個欄位裡可能有名字或姓氏。

Let's look at one of the sample values to better validate.

我們來看看其中一個範例值，以便更好地驗證。

Yep, it looks like User attributes is a JSON string, so it's easy for additional attributes to get added that do in fact look like names.

沒錯，看起來 User attributes 是一個 JSON 字串，所以很容易加入看起來像名字的額外屬性。

So we'll want to make sure that this is getting masked and we'll want to figure out how this got here in the first place.

所以我們要確保這個被遮蔽，並且要弄清楚它一開始是怎麼進來的。

To do that, I'll open up this table back in our Data Explorer view.

要做到這點，我會在我們的 Data Explorer 檢視中重新開啟這個表格。

Again, we can validate that this column is being properly masked and we can open up our lineage graph to see where it came from in the first place.

再次地，我們可以驗證這個欄位是否被正確遮蔽，並且可以開啟我們的 lineage 圖來看看它一開始是從哪裡來的。

Here's the table in question, as we can see by the red markings and the monitoring alert.

這是我們正在討論的表格，正如我們從紅色標記和監控警示中看到的。

But it's joining a lot of different tables from different systems.

但它正在串接來自不同系統的許多不同表格。

So let's expand this to see where the User Attributes column came from.

所以讓我們展開它，看看 User Attributes 欄位是從哪裡來的。

So now I know to reach out to the team that owns Postgres to see if this is expected or if further mitigation is needed.

所以現在我知道要聯繫擁有 Postgres 的團隊，看看這是否是預期的，或者是否需要進一步的緩解措施。

I can also follow the table downstream and see this floats into a feature store which powers our churn prediction model.

我也可以追蹤這個表格的下游，看看它如何流入一個 powers our churn prediction model 的 feature store。

If PII gets into that model, we are out of compliance.

如果 PII 進入那個模型，我們就會違反規定。

So I really want to roll this back to a version that was trained before the PII was introduced.

所以我想把這個回溯到一個在引入 PII 之前訓練的版本。

And to do that I'll just look at that model in Data Explorer.

要做到這一點，我會在 Data Explorer 中查看那個模型。

So I'll definitely want to roll back the churn prediction model to a version that was trained before PII was introduced.

所以，我絕對想把 churn prediction model 回溯到一個在引入 PII 之前訓練的版本。

In this case, we can see the latest version was trained after PII was introduced.

在這種情況下，我們可以看到最新版本是在引入 PII 之後訓練的。

So we'll want to go back to version one in our system, we're set up to serve the model that's Alias as Champion.

所以我們想回到我們系統中的版本一，我們已經設定好要服務被稱為 Champion 的模型。

So all we have to do to serve the previous version is Alias version one as champion.

所以我們要做的一切就是將版本一設為 Champion，以服務之前的版本。

Now we know we've successfully mitigated the model that we're serving, and we're now serving the SAFE model.

現在我們知道我們已經成功緩解了我們正在服務的模型，並且我們現在正在服務 SAFE 模型。

All in all, in the last couple of minutes, what we saw was that the system detected potential PII leakage.

總而言之，在過去幾分鐘裡，我們看到系統偵測到潛在的 PII 洩漏。

It automatically masked and mitigated any potential leakage to end users, and

它會自動遮罩並緩解任何潛在的洩漏給終端使用者，而且

it made it easy for us to understand where the PII was coming from and what potential systems and models were impacted downstream.

讓我們輕鬆瞭解 PII 的來源以及哪些潛在的系統和模型在下游受到影響。

Now that we've aliased the previous version as the champion, we know we're serving the SAFE model.

現在我們已經將先前版本別名為 Champion，我們知道我們正在服務 SAFE 模型。

In the last few minutes, we saw Enterprise Wide Reach with LakeHouse Federation with MySQL PostgreSQL and Snowflake available to access, manage and permission from within Unity Catalog.

在過去幾分鐘裡，我們看到透過 LakeHouse Federation 實現了企業級的廣泛觸及，MySQL、PostgreSQL 和 Snowflake 均可從 Unity Catalog 內進行存取、管理和權限設定。

We saw Governance for AI with our model Feature store and volumes directly available within Data Explorer.

我們看到了 AI 的治理，我們的模型 Feature store 和 volumes 直接可在 Data Explorer 中取得。

We also saw AI for Governance with Lakehouse Monitoring automatically detecting and resolving that potential PII issue.

我們也看到了治理的 AI，Lakehouse Monitoring 自動偵測並解決了潛在的 PII 問題。

This workflow has saved me tons of time compared to what it used to look like, and

這個工作流程為我節省了大量時間，與以前相比，而且

with that, I think maybe I deserve an extra long lunch.

說到這裡，我想我應該可以好好享受一頓豐盛的午餐。