Study guide
3.4 Unity Catalog Naming Conventions, Environment Isolation, & Data Granularity
Microsoft Certified: Azure Databricks Data Engineer Associate DP-750
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Azure Databricks Architecture & Compute Selection
1.1 DP-750 Exam Overview & Azure Databricks Lakehouse Architecture 1.2 Compute Types & Workload Matching 1.3 Cluster Access Modes, Standard/Dedicated, & Serverless Compute
2 Chapter 2: Compute Configuration, Performance & Cost Management
2.1 Autoscaling, Databricks Pools, & Node Type Selection 2.2 Photon Vectorized Engine & Cost Governance with Cluster Policies 2.3 Library Management: Workspace, Compute-Scoped, Notebook, & Init Scripts 2.4 Auto-Termination, Budget Alerts, & Tagging for Cost Allocation 2.5 Databricks Runtime & ML Runtime Selection, and Compute Access Permissions
3 Chapter 3: Unity Catalog Storage Infrastructure & Workspace Setup
3.1 Unity Catalog 3-Level Namespace & Metastore Architecture 3.2 Azure Managed Identity, Storage Credentials, & External Locations 3.3 Managed Tables, External Tables, & Unity Catalog Volumes 3.4 Unity Catalog Naming Conventions, Environment Isolation, & Data Granularity 3.5 Lakehouse Federation: Connections, Foreign Catalogs, & Query Pushdown
4 Chapter 4: Unity Catalog Privilege Model & Access Control
4.1 Hierarchical Privilege Model & Securable Objects 4.2 Account-Level Identities, SCIM Sync, & Group Management 4.3 Object Ownership, Ownership Transfer, & Administrative Delegation
5 Chapter 5: Advanced Data Security, Masking & ABAC
5.1 Dynamic Row Filters & Column Masking Implementation 5.2 Attribute-Based Access Control (ABAC) with Unity Catalog Tags 5.3 Data Isolation across Workspaces & Multiple Metastore Patterns 5.4 Secure Data Sharing within Azure Databricks
6 Chapter 6: Governance, Auditing, Lineage & Delta Sharing
6.1 Automated Data Lineage Tracking in Catalog Explorer 6.2 Unity Catalog System Tables: Audit, Billing, and Access Logs 6.3 Data Retention Policies, Time Travel Security, & Tag Governance 6.4 Delta Sharing Protocols: Open Sharing vs. Databricks-to-Databricks 6.5 Data Discovery: Table and Column Descriptions, and AI/BI Genie Instructions
7 Chapter 7: Ingestion with Auto Loader, Batch & Structured Streaming
7.1 Auto Loader (cloudFiles) Architecture, File Notification vs. Directory Listing 7.2 Auto Loader Schema Inference, Evolution, & Rescued Data Column 7.3 Batch Ingestion with COPY INTO, CTAS, & Multipart Files 7.4 Structured Streaming Fundamentals & Azure Event Hubs / Kafka Ingestion 7.5 Streaming Watermarking, Triggers, & Output Modes 7.6 Choosing an Ingestion Tool: Lakeflow Connect, Notebooks, & Azure Data Factory
8 Chapter 8: Delta Lake Core Mechanics, ACID & Table Maintenance
8.1 Delta Lake Architecture, Parquet Files, & the _delta_log Transaction Log 8.2 ACID Transactions, Concurrency Control, & Conflict Resolution 8.3 Compaction (OPTIMIZE), Z-Ordering, & Liquid Clustering (CLUSTER BY) 8.4 Data Retention, Snapshot Isolation, & VACUUM Operations 8.5 Delta Change Data Feed (CDF) & MERGE INTO Upsert Patterns 8.6 Choosing a Table Format: Delta, Parquet, CSV, JSON, & Apache Iceberg
9 Chapter 9: Data Cleansing, Transformation & Medallion Architecture
9.1 The Medallion Lakehouse Pattern: Bronze, Silver, and Gold Layer Design 9.2 Data Cleansing: Deduplication, Null Handling, & Type Casting in PySpark/SQL 9.3 Complex Transformations: Higher-Order Functions, Arrays, Structs, & JSON Parsing 9.4 Relational Joins, Window Functions, Set Operators, & Pivoting 9.5 Enforcing Data Quality: Check Constraints & Table Invariants 9.6 Data Profiling: Summary Statistics & Distribution Assessment
10 Chapter 10: Declarative Pipelines, Lakeflow Spark & Delta Live Tables
10.1 Declarative Pipeline Fundamentals: Streaming Tables vs. Materialized Views 10.2 Data Quality Expectations: expect, expect or drop, & expect or fail 10.3 Change Data Capture Ingestion with AUTO CDC / APPLY CHANGES INTO 10.4 Pipeline Execution Modes, Serverless DLT, & Event Logs
11 Chapter 11: Workflow Orchestration with Lakeflow Jobs
11.1 Multi-Task DAG Design in Lakeflow Jobs & Task Type Selection 11.2 Task Dependencies, Parameters, Task Values, & Dynamic Variables 11.3 Triggers (Scheduled, File Arrival, Continuous), Retries, & Timeouts 11.4 Notifications, Alerting (Email, Slack, Webhooks), & Failure Handling
12 Chapter 12: CI/CD, Git Integration & Databricks Asset Bundles
12.1 Databricks Git Folders & Source Control Integration 12.2 Databricks Asset Bundles (DAB) Architecture & databricks.yml Structure 12.3 Multi-Environment Deployment (Dev / Staging / Prod) & Target Overrides 12.4 Automated Testing, Unit Testing PySpark, & CI/CD Pipelines
13 Chapter 13: Workload Monitoring, Troubleshooting, & Performance Tuning
13.1 Spark UI Mastery: Query Execution DAG, Stages, Tasks, & Skew Analysis 13.2 Troubleshooting Common Failures: OOM, Disk Spill, Data Skew, & Network Timeouts 13.3 Performance Tuning: Adaptive Query Execution (AQE), Caching, & Broadcast Joins 13.4 Workspace Monitoring with Azure Monitor, Diagnostic Settings, & Log Analytics
3.4 Unity Catalog Naming Conventions, Environment Isolation, & Data Granularity
Key Takeaways Explain This Topic AI
Unity Catalog object names may not exceed 255 characters and may never contain a period, a space, a forward slash, an ASCII control character, or the DELETE character; Unity Catalog stores all object names in lowercase.
A hyphen is legal in a Unity Catalog name but forces every SQL reference to be backtick-escaped, which is why production naming standards use underscores exclusively.
The catalog is the isolation boundary: a catalog-per-environment layout (dev, staging, prod) plus workspace-catalog bindings prevents a development workspace from ever resolving a production table name.
Table granularity is the definition of what one row represents; declare it in the table COMMENT because grain determines the SCD type, the clustering keys, and whether an aggregate can ever be recomputed.
Granularity is a one-way door downstream: a finer grain can always be aggregated later, but a coarser grain permanently discards the detail needed to answer a new question.
Last updated: August 2026
3.4 Unity Catalog Naming Conventions, Environment Isolation, & Data Granularity
DP-750 Exam Focus: Two blueprint bullets meet here - "Apply naming conventions based on requirements, including isolation, development environment, and external sharing" and "Choose granularity on a column or table based on requirements." Both look like style questions and are actually architecture questions: names encode the isolation boundary, and grain encodes what the table can and cannot answer.
1. The Hard Rules Unity Catalog Enforces
Before any convention, these are constraints the platform itself imposes on all Unity Catalog object names:
- Rule Detail
- Maximum length 255 characters
Forbidden characters Period ( . ), space, forward slash ( / ), all ASCII control characters (hex 00-1F), and the DELETE character (hex 7F)
Case handling Unity Catalog stores all object names as lowercase . Column names preserve their casing, but queries against Unity Catalog tables are case-insensitive.
Escaping Names containing special characters such as a hyphen must be backtick-escaped in every SQL statement -- Legal, but every future reference needs backticks:
- CREATE CATALOG `retail-prod`;
- SELECT * FROM `retail-prod`.sales.orders;
- - Illegal: space and forward slash are rejected even inside backticks
- CREATE CATALOG `retail prod`; -- ERROR
-- The convention that avoids the problem entirely:
- CREATE CATALOG retail_prod;
- SELECT * FROM retail_prod.sales.orders;
Legacy contrast. Objects created in the legacy hive_metastore catalog are stricter still: schema, table, view, and function names there may contain only alphanumeric ASCII characters and underscores, and anything else raises INVALID_SCHEMA_OR_RELATION_NAME . Unity Catalog is more permissive, which is exactly why an explicit convention matters.
Delta Lake Column-Name Restriction
Delta tables without column mapping ( delta.columnMapping.mode = name ) cannot have column names containing a space, comma, semicolon, curly brace, parenthesis, newline, tab, or equals sign. Enabling column mapping lifts the restriction but upgrades the table protocol - which, as Section 2.5 notes, can lock out older readers.
2. A Naming Standard That Encodes Isolation
Unity Catalog's three-level namespace ( catalog.schema.object ) gives you exactly one strong isolation boundary - the catalog - and one organizational boundary - the schema . A working convention:
- Object Pattern Example
- Catalog (environment-first) {env}_{domain} prod_retail , dev_retail , stg_retail
- Schema (medallion layer or subject) {layer} or {subject} bronze , silver , gold_finance
- Table {entity} or {entity}_{grain} orders , orders_daily_agg
- Streaming/CDC target {entity}_scd2 or {entity}_current customers_scd2
- Volume {purpose}_{source} landing_pos_files , checkpoints_ingest
- External location {env}_{storage_account}_{container} prod_adlsretail_bronze
- Storage credential {env}_{access_connector} prod_mi_retail
- Connection (federation) {source_system}_{env} sqlserver_orders_prod
- Delta Sharing share {provider}_{consumer}_{subject} retail_partnerco_sales
- Delta Sharing recipient {external_org} partnerco
- Account group {env}_{domain}_{role} prod_retail_readers , prod_retail_engineers
- Why Environment Belongs in the Catalog Name
Putting the environment in the catalog rather than the schema or the table gives you three properties at once:
Fully qualified names differ across environments , so a notebook promoted from dev to prod fails loudly if someone forgot to parameterize the catalog, instead of silently writing dev data into a prod table.
Privileges are grantable at the isolation boundary. GRANT USE CATALOG ON CATALOG prod_retail TO prod_retail_readers scopes an entire environment in one statement.
Workspace bindings become meaningful. A catalog can be bound to specific workspaces so that prod_retail is not even resolvable from the development workspace: -- Restrict a catalog to named workspaces (also available in Catalog Explorer -> Workspaces)
ALTER CATALOG prod_retail SET WORKSPACE BINDINGS ('<prod-workspace-id>');
Combine this with a Databricks Asset Bundle target variable so the catalog name is injected per environment rather than hardcoded (see Section 12.3).
Naming for External Sharing
Delta Sharing objects are read by people outside your organization, so their names leak information. Two rules:
Never encode internal system names, cost centers, or project codenames in share and recipient names - a share called finco_project_bluebird_m_and_a tells a partner more than intended.
Name the relationship, not the storage. retail_partnerco_sales is stable when the underlying table moves; adls_prod_container3_export is not.
3. Choosing Table Granularity (Grain)
Grain is the answer to one sentence: "one row in this table represents exactly one ____." If you cannot finish that sentence in a few words, the table has no defined grain and its aggregates cannot be trusted.
Layer Typical grain Why
Bronze One row per source record or event, as delivered, with ingest metadata Preserves replay ability; never aggregate in bronze
Silver One row per business entity or per validated transaction Deduplicated, conformed, type-cast, joined to reference data
Gold One row per reporting grain, e.g. one row per store per day Pre-aggregated for BI latency; grain matches the dashboard
The One-Way Door
A finer grain can always be rolled up later. A coarser grain permanently destroys the detail needed to answer a question nobody has asked yet. If a gold table stores revenue per store per day, no query can ever recover revenue per hour. Practically:
Keep bronze and silver at the finest grain the source provides.
Aggregate only in gold, and only to a grain a stakeholder has actually requested.
Build additional gold tables at different grains rather than coarsening an existing one.
Column Granularity
The same decision applies within a column:
- Column Coarser Finer Trade-off
- Time order_date (DATE) order_timestamp (TIMESTAMP) A DATE column cannot support hourly SLA reporting
- Geography country_code postal_code Postal-level detail may be personally identifying and require a column mask
- Product category_id sku SKU explodes cardinality but enables assortment analysis
Money revenue_usd rounded revenue_usd DECIMAL(18,4) Rounding early makes reconciliation with source systems impossible
Finer column granularity raises cardinality , which is exactly the input to the clustering decision in Section 8.3: high-cardinality, high-selectivity filter columns are good liquid clustering keys, while low-cardinality columns are usually better left unclustered.
Declare the Grain in Metadata
A grain that lives only in a design document is a grain that will be violated. Put it where every consumer and every discovery tool can read it: COMMENT ON TABLE prod_retail.gold.store_sales_daily IS
- 'Grain: one row per store_id per business_date. Revenue is net of returns. Late-arriving
- sales are restated for 7 days after business_date.';
- ALTER TABLE prod_retail.gold.store_sales_daily
- ALTER COLUMN business_date COMMENT 'Store local calendar date, not UTC ingest date.';
Section 6.5 shows how these descriptions feed Catalog Explorer search and AI/BI Genie.
4. Exam Traps
"Add the environment to the table name" ( orders_prod ) is the wrong answer. Environment belongs in the catalog so it can be bound and granted.
Backticks do not rescue a space or a period. Those characters are rejected outright; only characters like hyphens are merely inconvenient.
Aggregating in silver to "save storage" is a grain violation; the correct answer is a gold table at the requested grain.
Case sensitivity : Unity Catalog lowercases object names, so Orders and orders are the same table. Column casing is preserved but queries are still case-insensitive.