UNITY CATALOG
DAY 9
WHAT IS UNITY CATALOG
Unity Catalog provides centralized access control, auditing, lineage, and data
discovery capabilities across Azure Databricks workspaces.
Key features of Unity Catalog include:
• Define once, secure everywhere: Unity Catalog offers a single place to administer
data access policies that apply across all workspaces.
• Standards-compliant security model: Unity Catalog’s security model is based on
standard ANSI SQL and allows administrators to grant permissions in their existing data
lake using familiar syntax, at the level of catalogs, databases (also called schemas),
tables, and views.
• Built-in auditing and lineage: Unity Catalog automatically captures user-level audit
logs that record access to your data. Unity Catalog also captures lineage data that
tracks how data assets are created and used across all languages.
• Data discovery: Unity Catalog lets you tag and document data assets, and provides a
search interface to help data consumers find data.
• System tables (Public Preview): Unity Catalog lets you easily access and query
your account’s operational data, including audit logs, billable usage, and lineage.
The Unity Catalog object
model
The Unity Catalog object
model
•Metastore: The top-level container for metadata. Each metastore exposes a three-level namespace (catalog.schema.table) that organizes your data.
•Catalog: The first layer of the object hierarchy, used to organize your data assets.
•Schema: Also known as databases, schemas are the second layer of the object hierarchy and contain tables and views.
•Volume: Volumes sit alongside tables and views at the lowest level of the object hierarchy and provide governance for non-tabular data.
•Table: At the lowest level in the object hierarchy are tables and views.
Enable: DBFS Location
Unity Catalog Configuration - Prerequisites
1. Create a new user in azure active directory
Enable DBFS LOCATION
Sample datasets
https://learn.microsoft.com/en-us/azure/databricks/getting-started/qu
ick-start
https://learn.microsoft.com/en-us/azure/databricks/dbfs/databricks-d
atasets
https://www.kaggle.com/datasets/shivam2503/diamonds/
Databricks Offering Some SAMPLE
DATASET
Scheme:
Table Selection
Detailed Description:
https://learn.microsoft.com/en-us/azure/databricks/data/
DAY 2
Databricks unity catalog,azure
mounting
https://stevedem.github.io/mounting-adls-gen2-in-databricks/
https://learn.microsoft.com/en-us/azure/databricks/dbfs/mounts
https://learn.microsoft.com/en-us/azure/databricks/data-governance/unity-catalog/b
est-practices
Login from given site https://accounts.azuredatabricks.net
Click on the Workspace menu
Data Tab: Helps to create metastore
User management:
Create a Unity catalog
metastore
