Study guide
4.1 Hierarchical Privilege Model & Securable Objects
Microsoft Certified: Azure Databricks Data Engineer Associate DP-750
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Azure Databricks Architecture & Compute Selection
1.1 DP-750 Exam Overview & Azure Databricks Lakehouse Architecture 1.2 Compute Types & Workload Matching 1.3 Cluster Access Modes, Standard/Dedicated, & Serverless Compute
2 Chapter 2: Compute Configuration, Performance & Cost Management
2.1 Autoscaling, Databricks Pools, & Node Type Selection 2.2 Photon Vectorized Engine & Cost Governance with Cluster Policies 2.3 Library Management: Workspace, Compute-Scoped, Notebook, & Init Scripts 2.4 Auto-Termination, Budget Alerts, & Tagging for Cost Allocation 2.5 Databricks Runtime & ML Runtime Selection, and Compute Access Permissions
3 Chapter 3: Unity Catalog Storage Infrastructure & Workspace Setup
3.1 Unity Catalog 3-Level Namespace & Metastore Architecture 3.2 Azure Managed Identity, Storage Credentials, & External Locations 3.3 Managed Tables, External Tables, & Unity Catalog Volumes 3.4 Unity Catalog Naming Conventions, Environment Isolation, & Data Granularity 3.5 Lakehouse Federation: Connections, Foreign Catalogs, & Query Pushdown
4 Chapter 4: Unity Catalog Privilege Model & Access Control
4.1 Hierarchical Privilege Model & Securable Objects 4.2 Account-Level Identities, SCIM Sync, & Group Management 4.3 Object Ownership, Ownership Transfer, & Administrative Delegation
5 Chapter 5: Advanced Data Security, Masking & ABAC
5.1 Dynamic Row Filters & Column Masking Implementation 5.2 Attribute-Based Access Control (ABAC) with Unity Catalog Tags 5.3 Data Isolation across Workspaces & Multiple Metastore Patterns 5.4 Secure Data Sharing within Azure Databricks
6 Chapter 6: Governance, Auditing, Lineage & Delta Sharing
6.1 Automated Data Lineage Tracking in Catalog Explorer 6.2 Unity Catalog System Tables: Audit, Billing, and Access Logs 6.3 Data Retention Policies, Time Travel Security, & Tag Governance 6.4 Delta Sharing Protocols: Open Sharing vs. Databricks-to-Databricks 6.5 Data Discovery: Table and Column Descriptions, and AI/BI Genie Instructions
7 Chapter 7: Ingestion with Auto Loader, Batch & Structured Streaming
7.1 Auto Loader (cloudFiles) Architecture, File Notification vs. Directory Listing 7.2 Auto Loader Schema Inference, Evolution, & Rescued Data Column 7.3 Batch Ingestion with COPY INTO, CTAS, & Multipart Files 7.4 Structured Streaming Fundamentals & Azure Event Hubs / Kafka Ingestion 7.5 Streaming Watermarking, Triggers, & Output Modes 7.6 Choosing an Ingestion Tool: Lakeflow Connect, Notebooks, & Azure Data Factory
8 Chapter 8: Delta Lake Core Mechanics, ACID & Table Maintenance
8.1 Delta Lake Architecture, Parquet Files, & the _delta_log Transaction Log 8.2 ACID Transactions, Concurrency Control, & Conflict Resolution 8.3 Compaction (OPTIMIZE), Z-Ordering, & Liquid Clustering (CLUSTER BY) 8.4 Data Retention, Snapshot Isolation, & VACUUM Operations 8.5 Delta Change Data Feed (CDF) & MERGE INTO Upsert Patterns 8.6 Choosing a Table Format: Delta, Parquet, CSV, JSON, & Apache Iceberg
9 Chapter 9: Data Cleansing, Transformation & Medallion Architecture
9.1 The Medallion Lakehouse Pattern: Bronze, Silver, and Gold Layer Design 9.2 Data Cleansing: Deduplication, Null Handling, & Type Casting in PySpark/SQL 9.3 Complex Transformations: Higher-Order Functions, Arrays, Structs, & JSON Parsing 9.4 Relational Joins, Window Functions, Set Operators, & Pivoting 9.5 Enforcing Data Quality: Check Constraints & Table Invariants 9.6 Data Profiling: Summary Statistics & Distribution Assessment
10 Chapter 10: Declarative Pipelines, Lakeflow Spark & Delta Live Tables
10.1 Declarative Pipeline Fundamentals: Streaming Tables vs. Materialized Views 10.2 Data Quality Expectations: expect, expect or drop, & expect or fail 10.3 Change Data Capture Ingestion with AUTO CDC / APPLY CHANGES INTO 10.4 Pipeline Execution Modes, Serverless DLT, & Event Logs
11 Chapter 11: Workflow Orchestration with Lakeflow Jobs
11.1 Multi-Task DAG Design in Lakeflow Jobs & Task Type Selection 11.2 Task Dependencies, Parameters, Task Values, & Dynamic Variables 11.3 Triggers (Scheduled, File Arrival, Continuous), Retries, & Timeouts 11.4 Notifications, Alerting (Email, Slack, Webhooks), & Failure Handling
12 Chapter 12: CI/CD, Git Integration & Databricks Asset Bundles
12.1 Databricks Git Folders & Source Control Integration 12.2 Databricks Asset Bundles (DAB) Architecture & databricks.yml Structure 12.3 Multi-Environment Deployment (Dev / Staging / Prod) & Target Overrides 12.4 Automated Testing, Unit Testing PySpark, & CI/CD Pipelines
13 Chapter 13: Workload Monitoring, Troubleshooting, & Performance Tuning
13.1 Spark UI Mastery: Query Execution DAG, Stages, Tasks, & Skew Analysis 13.2 Troubleshooting Common Failures: OOM, Disk Spill, Data Skew, & Network Timeouts 13.3 Performance Tuning: Adaptive Query Execution (AQE), Caching, & Broadcast Joins 13.4 Workspace Monitoring with Azure Monitor, Diagnostic Settings, & Log Analytics
4.1 Hierarchical Privilege Model & Securable Objects
Key Takeaways Explain This Topic AI
Unity Catalog organizes all data and operational assets into a strict securable object hierarchy: Metastore -> Catalog -> Schema -> Securables (Tables, Views, Volumes, Functions, Models).
Permissions cascade downward through privilege inheritance: granting an operational privilege on a parent container (Catalog or Schema) automatically propagates that privilege to all current and future child objects within it.
Querying or modifying any table, view, or volume strictly requires traversal privileges: USE CATALOG on the parent catalog AND USE SCHEMA on the parent schema, in addition to the object-level action privilege (such as SELECT, MODIFY, or READ VOLUME).
Unity Catalog operates on an explicit, default-deny security model managed via standard ANSI SQL GRANT and REVOKE syntax against users, service principals, and account groups.
The ALL PRIVILEGES grant assigns all current and future operational privileges on a securable object to a principal, but it does NOT grant object ownership or the ability to alter ownership.
Last updated: August 2026
4.1 Hierarchical Privilege Model & Securable Objects
DP-750 Exam Focus: Master the Unity Catalog hierarchical securable object model and the mechanics of downward privilege inheritance. You must understand the prerequisite traversal privileges ( USE CATALOG on the catalog and USE SCHEMA on the schema) required before a user can query tables or read volumes, the exact ANSI SQL GRANT and REVOKE syntax, and the operational differences between ALL PRIVILEGES , specific verb privileges, and object ownership.
The Unity Catalog Securable Object Hierarchy
In Azure Databricks Unity Catalog, a securable object is any entity on which permissions can be granted to a principal (user, service principal, or account group). Unity Catalog structures all securable objects in a strict, containment-based tree hierarchy. Access control applied at higher tiers of the tree influences accessibility at lower tiers. +---------------------------------------------------+
- | METASTORE |
- | (Top-level account container for metadata) |
- +---------------------------------------------------+
- |
- +-------------------------------------+-------------------------------------+
- | | |
- v v v
- +---------------------------------+ +-----------------------------------+ +---------------------------------+
- | STORAGE CREDENTIALS | | CATALOGS | | EXTERNAL LOCATIONS |
- | (Encapsulates Azure Managed Id) | | (First level of 3-level name) | | (Gov. ADLS Gen2 URI paths) |
- +---------------------------------+ +-----------------------------------+ +---------------------------------+
- | | |
- | (Used to create) v | (Used to register)
- +-----------------------------> SCHEMAS <------------------------------+
- (Second level / database)
- |
- +---------------------+-------------------------+-------------------------+---------------------+
- | | | | |
- v v v v v
- +-----------------+ +-----------------+ +-----------------+ +-----------------+ +-----------------+
- | TABLES | | VIEWS | | VOLUMES | | FUNCTIONS | | MODELS |
- | (Managed/Ext.) | | (Standard/Mask) | | (Managed/Ext.) | | (UDFs / TVFs) | | (MLflow Models) |
- +-----------------+ +-----------------+ +-----------------+ +-----------------+ +-----------------+
- Primary Securable Object Types
Metastore: The root metadata container deployed per Azure region. Governs global objects such as Catalogs, Storage Credentials, External Locations, Connections, Delta Sharing Providers, and Recipients.
Catalog: The primary organizational boundary (the first identifier in catalog.schema.table ). Represents environments (e.g., prod_dw , dev_sandbox ) or business domains ( finance , marketing ).
Schema (Database): The logical grouping within a catalog (the second identifier in catalog.schema.table ). Represents business processes or medallion layers ( bronze , silver , gold ).
Data Securables (Leaf / Execution level):
Tables: Tabular datasets (Managed Delta tables or External tables).
Views: Virtual tables defined on top of one or more tables or views.
Volumes: Logical governance endpoints for unstructured or semi-structured files in ADLS Gen2.
Functions: User-defined scalar and table-valued functions (UDFs/TVFs) registered in the catalog.
Models: Registered machine learning models tracked and deployed via Unity Catalog MLflow.
Infrastructure Securables:
Storage Credentials: Authentication mechanisms wrapping Azure Access Connectors / Managed Identities.
External Locations: Governed bindings between Storage Credentials and specific ADLS Gen2 paths ( abfss://... ).
Connections: Governed connection endpoints to external database engines (Lakehouse Federation).
Privilege Inheritance Mechanics
Unity Catalog implements downward privilege inheritance . When an operational privilege is granted on a parent securable object, that privilege is automatically and implicitly inherited by all existing and future child objects contained within that parent container. +------------------------------------------+
- | GRANT SELECT ON CATALOG | <-- Assigned at Catalog level
- | prod_catalog | to group `analysts`
- +------------------------------------------+
- |
- | (Inherited automatically)
- v
- +------------------------------------------+
- | All Schemas | <-- `analysts` can query tables
- | (e.g., prod_catalog.silver, etc.) | in all schemas
- +------------------------------------------+
- |
- | (Inherited automatically)
- v
- +------------------------------------------+
- | All Current & Future Tables | <-- Newly created tables in these
- | (e.g., prod_catalog.silver.orders_fact) | schemas are instantly queryable
- +------------------------------------------+
- How Inheritance Operates in Practice
Catalog-Level Grants: Executing GRANT SELECT ON CATALOG prod_dw TO data_analysts; gives data_analysts read access to every table and view across every schema in prod_dw . When a data engineer adds a new schema or creates a new table in prod_dw weeks later, data_analysts immediately inherits SELECT privileges on that new asset without administrative intervention.
Schema-Level Grants: Executing GRANT SELECT ON SCHEMA prod_dw.gold TO marketing_team; restricts inherited read access strictly to tables and views within the gold schema of prod_dw . Assets in prod_dw.silver or prod_dw.bronze remain inaccessible.
Object-Level Grants: Executing GRANT SELECT ON TABLE prod_dw.gold.customer_360 TO external_auditors; grants access strictly to that single table.
Exam Tip: Privilege inheritance in Unity Catalog flows only downward , never upward. Granting SELECT on a table does not grant privileges on its parent schema or catalog. Furthermore, Unity Catalog uses a default-deny model; if no grant exists at the object, schema, catalog, or metastore level, access is denied.
The Traversal Prerequisite Principle: USE CATALOG and USE SCHEMA
A critical concept tested heavily on the DP-750 exam is the requirement for traversal privileges . In Unity Catalog, granting an action privilege (such as SELECT on a table or READ VOLUME on a volume) is insufficient by itself for a user to interact with the object. The user must also possess traversal permissions along the full parent hierarchy.
The Mandatory Traversal Chain for Tables and Views
To execute SELECT * FROM prod_dw.silver.orders; , the querying principal must hold:
USE CATALOG on prod_dw (the parent catalog).
USE SCHEMA on prod_dw.silver (the parent schema).
SELECT on prod_dw.silver.orders (or inherit SELECT from silver , prod_dw , or the metastore). Required Privilege Check Flow for: SELECT * FROM prod_dw.silver.orders
+------------------------+
| 1. Check Catalog Level | ---> Does principal have USE CATALOG on `prod_dw`?
- +------------------------+ [NO] --> FAIL: Permission Denied (Cannot traverse catalog)
- | [YES]
- v
- +------------------------+
| 2. Check Schema Level | ---> Does principal have USE SCHEMA on `prod_dw.silver`?
- +------------------------+ [NO] --> FAIL: Permission Denied (Cannot traverse schema)
- | [YES]
- v
- +------------------------+
| 3. Check Object Level | ---> Does principal have SELECT on `prod_dw.silver.orders`?
- +------------------------+ (Or inherited SELECT from schema/catalog?)
- | [YES] [NO] --> FAIL: Permission Denied (Cannot read table)
- v
- [SUCCESS: Query Executes]
- What Traversal Privileges Enable
Privilege Verb Securable Target What It Allows What It Does NOT Allow
USE CATALOG CATALOG Allows the user to traverse into the catalog, inspect schema names (if permitted), and resolve 3-level names. Does NOT allow reading data inside tables, creating schemas, or modifying catalog metadata.
USE SCHEMA SCHEMA Allows the user to traverse into the schema, resolve 2-level/1-level object names, and execute functions. Does NOT allow reading table data ( SELECT ), creating tables ( CREATE TABLE ), or reading files ( READ VOLUME ).
BROWSE CATALOG , SCHEMA , TABLE Allows the user to view object metadata and schema definitions in Catalog Explorer without granting data access. Does NOT allow reading data or traversing the namespace in SQL queries. -- Correct Traversal Provisioning for an Analyst Group
- GRANT USE CATALOG ON CATALOG prod_dw TO `bi_analysts`;
- GRANT USE SCHEMA ON SCHEMA prod_dw.gold TO `bi_analysts`;
- GRANT SELECT ON TABLE prod_dw.gold.daily_revenue TO `bi_analysts`;
-- Alternative using Inheritance at the Schema level:
- GRANT USE CATALOG ON CATALOG prod_dw TO `bi_analysts`;
- GRANT USE SCHEMA, SELECT ON SCHEMA prod_dw.gold TO `bi_analysts`;
- Complete Unity Catalog Privilege Matrix
Unity Catalog supports specific ANSI SQL privilege verbs tailored to each securable object type:
Securable Object Valid Privileges Description & Primary Use Case
METASTORE CREATE CATALOG Allows principal to create top-level catalogs in the metastore.
CREATE STORAGE CREDENTIAL Allows principal to create storage credentials wrapping Azure Access Connectors.
CREATE EXTERNAL LOCATION Allows principal to register new external ADLS Gen2 locations.
CREATE CONNECTION Allows principal to configure Lakehouse Federation connection endpoints.
MANAGE ALLOWLIST Allows principal to configure init script, JAR, and library allowlists.
CATALOG USE CATALOG Required traversal privilege to access schemas and securables within the catalog.
CREATE SCHEMA Allows principal to create new schemas inside the catalog.
BROWSE Allows inspecting catalog structure in Catalog Explorer without data access.
APPLY TAG Allows adding or modifying Unity Catalog tags on the catalog.
ALL PRIVILEGES Grants all available catalog-level operational privileges.
SCHEMA USE SCHEMA Required traversal privilege to access tables, volumes, and functions in the schema.
CREATE TABLE Allows creating managed and external tables within the schema.
CREATE VIEW Allows creating standard or dynamic views within the schema.
CREATE VOLUME Allows creating managed or external volumes within the schema.
CREATE FUNCTION Allows registering SQL UDFs and custom functions in the schema.
CREATE MODEL Allows logging and registering MLflow models in the schema.
ALL PRIVILEGES Grants all available schema-level operational privileges.
TABLE SELECT Allows reading data from the table (Delta, Parquet, CSV).
MODIFY Allows inserting, updating, deleting, and merging data ( INSERT , UPDATE , DELETE , MERGE ).
APPLY TAG Allows tagging table columns and table metadata for governance/ABAC.
ALL PRIVILEGES Grants full operational data control ( SELECT , MODIFY , APPLY TAG ).
VIEW SELECT Allows reading and executing the view definition.
ALL PRIVILEGES Grants full operational control over the view.
VOLUME READ VOLUME Allows reading files stored within the managed or external volume.
WRITE VOLUME Allows writing, appending, and deleting files within the volume.
ALL PRIVILEGES Grants both READ VOLUME and WRITE VOLUME .
FUNCTION EXECUTE Allows invoking the scalar, table-valued, or Python UDF in SQL/DataFrame queries.
EXTERNAL LOCATION CREATE EXTERNAL TABLE Allows creating external tables pointing to ADLS Gen2 paths within this location.
CREATE EXTERNAL VOLUME Allows creating external volumes pointing to ADLS Gen2 paths within this location.
READ FILES Allows querying files directly via SELECT * FROM text. abfss://...``.
WRITE FILES Allows writing files directly to ADLS Gen2 storage paths.
STORAGE CREDENTIAL CREATE EXTERNAL LOCATION Allows binding this credential to create external locations.
ANSI SQL Syntax: GRANT , REVOKE , and SHOW GRANTS
Unity Catalog uses standard ANSI SQL statements to manage privileges. All grant statements can be executed in Databricks Notebooks, the SQL Editor, or programmatically via the Databricks REST API and Terraform.
1. Granting Privileges -- Syntax: GRANT <privileges> ON <securable_type> <securable_name> TO <principal>;
- - Granting traversal and read access on a catalog and schema
- GRANT USE CATALOG ON CATALOG finance_prod TO `finance_analysts`;
- GRANT USE SCHEMA, SELECT ON SCHEMA finance_prod.reporting TO `finance_analysts`;
- - Granting write privileges on a table
- GRANT MODIFY ON TABLE finance_prod.reporting.monthly_gl TO `finance_data_engineers`;
- - Granting file access privileges on a Volume
- GRANT READ VOLUME, WRITE VOLUME ON VOLUME finance_prod.raw.invoices_landing TO `ingestion_pipeline_sp`;
- - Granting external storage creation on an External Location
- GRANT CREATE EXTERNAL TABLE, READ FILES ON EXTERNAL LOCATION adls_landing_loc TO `data_engineers`;
- 2. Revoking Privileges -- Syntax: REVOKE <privileges> ON <securable_type> <securable_name> FROM <principal>;
- - Revoke write access from a group
- REVOKE MODIFY ON TABLE finance_prod.reporting.monthly_gl FROM `finance_data_engineers`;
- - Revoke schema creation from a user
- REVOKE CREATE SCHEMA ON CATALOG finance_prod FROM `contractor@company.com`;
- 3. Inspecting Active Grants ( SHOW GRANTS ) -- Show all privileges granted on a specific table
- SHOW GRANTS ON TABLE finance_prod.reporting.monthly_gl;
- - Show all privileges granted to a specific principal across securables
- SHOW GRANTS TO `finance_analysts`;
- - Show grants on an external location
- SHOW GRANTS ON EXTERNAL LOCATION adls_landing_loc;
- ALL PRIVILEGES vs. Object Ownership
A critical distinction on the DP-750 exam is the difference between having ALL PRIVILEGES on an object versus being the Owner of the object.
- Capability / Action Principal with ALL PRIVILEGES Object Owner ( OWNER ) Metastore Admin
- Query / Read Data ( SELECT , READ VOLUME ) Yes Yes Yes
- Modify Data ( MODIFY , WRITE VOLUME ) Yes Yes Yes
- Alter Table Schema / Add Columns ( ALTER TABLE ) No (Requires Ownership) Yes Yes
- Drop the Object ( DROP TABLE , DROP SCHEMA ) No (Requires Ownership) Yes Yes
- Grant / Revoke Privileges to Other Principals No (Requires Ownership) Yes Yes
- Transfer Ownership ( ALTER ... OWNER TO ) No (Requires Ownership) Yes Yes
- Set / Modify Object Properties / Tags Yes (via APPLY TAG ) Yes Yes
Key Concept: ALL PRIVILEGES is an operational shortcut. It grants all current and future operational verbs (e.g., SELECT , MODIFY , CREATE TABLE ) applicable to that object type. However, ALL PRIVILEGES does NOT confer administrative control . You cannot alter object definitions, drop objects, or delegate permissions to others unless you are the explicit Owner of the object or a Metastore Admin.