ExamFinch.com My Study

Study guide

1.5 Databricks SQL and the Photon Engine

Practice this exam

Databricks Certified Data Engineer Associate Exam

Track your progress as you read

Finished sections get a check mark.

1 Introduction

About the Databricks Certified Data Engineer Associate Exam Study Plan and Strategies

2 Domain 1: Databricks Intelligence Platform (10%)

1.1 The Lakehouse Architecture 1.2 Databricks Workspace and Components 1.3 Delta Lake Fundamentals 1.4 Databricks Repos, Secrets, and Development Tools 1.5 Databricks SQL and the Photon Engine 1.6 Databricks Runtime and Compute Configuration

3 Domain 2: Development and Ingestion (30%)

2.1 Auto Loader (cloudFiles) 2.2 Reading Data with Spark SQL and PySpark 2.3 Writing Data to Delta Tables 2.4 Spark SQL Query Fundamentals 2.5 Complex Data Types and Higher-Order Functions 2.6 User-Defined Functions (UDFs) 2.7 Structured Streaming Fundamentals 2.8 Databricks Notebooks and Code Execution 2.9 Data Extraction from External Sources 2.10 Table Management and Properties

4 Domain 3: Data Processing & Transformations (31%)

3.1 The Medallion Architecture (Bronze, Silver, Gold) 3.2 Delta Lake Optimization: OPTIMIZE, VACUUM, and Liquid Clustering 3.3 Advanced SQL Transformations 3.4 Incremental Data Processing Patterns 3.5 Lakeflow Declarative Pipelines (Delta Live Tables) 3.6 Implementing a Multi-Hop Pipeline 3.7 Slowly Changing Dimensions (SCD) Patterns 3.8 Data Deduplication Strategies

5 Domain 4: Productionizing Data Pipelines (18%)

4.1 Databricks Workflows and Lakeflow Jobs 4.2 Databricks Asset Bundles (Declarative Automation Bundles) 4.3 Pipeline Monitoring, Alerting, and Error Handling 4.4 Testing and Deployment Best Practices 4.5 Pipeline Performance and Cost Optimization 4.6 Designing Idempotent and Recoverable Pipelines

6 Domain 5: Data Governance & Quality (11%)

5.1 Unity Catalog: Architecture and Three-Level Namespace 5.2 Access Control and Permissions 5.3 Data Lineage and Audit Logging 5.4 Delta Sharing 5.5 Lakehouse Federation 5.6 Data Quality Best Practices 5.7 Row-Level and Column-Level Security 5.8 Compliance and PII Data Management

1.5 Databricks SQL and the Photon Engine

Key Takeaways Explain This Topic AI

Databricks SQL provides a warehouse-style experience: the SQL editor, queries, dashboards, alerts, and BI-tool connectivity over lakehouse tables.

SQL warehouses are the compute for Databricks SQL and come in Classic, Pro, and Serverless tiers with differing startup speed and features.

Serverless SQL warehouses start in seconds and use Intelligent Workload Management (IWM) for AI-driven autoscaling.

Photon is Databricks' native C++ vectorized engine that processes data in columnar batches to accelerate SQL and DataFrame workloads.

Photon is always on for serverless and SQL warehouses, and is enabled by default but toggleable on classic clusters.

Last updated: June 2026

Databricks SQL

Databricks SQL (DBSQL) brings the experience of a data warehouse to the lakehouse. It gives analysts and engineers a dedicated environment to run SQL against governed Delta tables without touching Spark notebooks. Its components include:

SQL editor — write and run queries with autocomplete and result visualizations.

Queries — saved, parameterized SQL that can be scheduled to refresh.

Dashboards — collections of visualizations built from queries, refreshed on a schedule.

Alerts — trigger notifications when a query result crosses a threshold.

BI connectivity — native connectors and JDBC/ODBC let Power BI, Tableau, and Looker query lakehouse tables directly.

All of this runs on Unity Catalog-governed data, so the same permissions and lineage apply whether data is accessed from SQL, a notebook, or a BI tool.

SQL Warehouses

A SQL warehouse is the compute that powers Databricks SQL — analogous to a cluster but tuned for SQL/BI concurrency. There are three tiers:

  • Warehouse type Startup Notes
  • Classic Slower (minutes) Runs in your cloud account; basic DBSQL features
  • Pro Moderate Adds advanced features; still in your account
  • Serverless Seconds (2-6s) Compute managed by Databricks; instant scaling

Serverless SQL warehouses start in seconds and use Intelligent Workload Management (IWM) to autoscale across queries and concurrency automatically. They support all three DBSQL performance features — Photon , Predictive I/O , and IWM — and remove almost all infrastructure management. For exam purposes: serverless = fastest startup and least management; classic = slowest, runs in your own cloud subscription.

The Photon Engine

Photon is Databricks' native vectorized query engine , written in C++ , that accelerates analytical workloads. Instead of executing through the JVM-based Spark SQL engine row by row, Photon processes data in columnar batches using vectorized, SIMD-friendly operations — dramatically speeding up scans, joins, aggregations, and writes.

Key facts to remember:

Photon replaces the JVM execution engine for supported operations with a native C++ runtime; it is fully compatible with the Spark and DataFrame APIs, so no code changes are needed.

It accelerates SQL workloads, DataFrame API calls, ETL pipelines, and stateless streaming .

It is a drop-in accelerator : where an operation is not yet supported by Photon, execution falls back to standard Spark transparently.

Photon can reduce DBU cost per query even though its compute carries a higher DBU rate, because jobs finish faster.

Where Photon Runs

  • Compute Photon status
  • Serverless compute Always enabled
  • SQL warehouses Always enabled
  • Serverless Lakeflow Declarative Pipelines Always enabled
  • Classic all-purpose / jobs compute Enabled by default, toggleable

Because SQL warehouses always run Photon, Databricks SQL queries benefit from its acceleration automatically. On classic clusters, Photon is on by default but can be turned off — a useful exam distinction. Together, Databricks SQL + Photon give the lakehouse warehouse-class query performance directly on open Delta tables, eliminating the need to move data into a separate analytics warehouse.

Performance Features Behind Databricks SQL

Serverless SQL warehouses bundle three engine features that work together, and the exam expects you to recognize each by name:

Photon — the native C++ vectorized engine that executes the query plan in columnar batches.

Predictive I/O — uses machine learning to read only the data needed for a query, accelerating selective reads and point lookups (especially with deletion vectors) so the engine scans far fewer bytes.

Intelligent Workload Management (IWM) — AI-driven autoscaling that predicts query demand and adds or removes warehouse clusters to meet concurrency without manual tuning.

These combine so a dashboard hitting a serverless warehouse gets fast startup, automatic scaling under load, and Photon-accelerated execution with no configuration.

How DBSQL Fits the Lakehouse

Databricks SQL is the warehouse persona of the lakehouse. Analysts who only know SQL get a familiar editor, scheduled queries, dashboards, and alerts, while connecting BI tools through JDBC/ODBC, all reading the same governed Delta tables that data engineers populate with notebooks and Lakeflow pipelines. Because Unity Catalog enforces permissions and lineage uniformly, a column an engineer masks is masked for the BI tool too — there is no separate warehouse copy with its own, drifting security model.

Exam-Critical Distinctions

SQL warehouse vs cluster: a SQL warehouse powers Databricks SQL and BI concurrency; a cluster runs notebooks and jobs. Both are compute, but you would not run a notebook on a SQL warehouse.

Serverless vs classic: serverless starts in seconds and is managed by Databricks; classic runs in your own cloud account and starts in minutes.

Photon is automatic on SQL warehouses , so DBSQL queries are accelerated whether or not the user knows Photon exists. Knowing where Photon is always-on versus toggleable is a frequently tested point. Remember too that Photon requires no code changes — it transparently accelerates the same SQL and DataFrame operations you already write and silently falls back to standard Spark for any operation it does not yet support, so enabling it is purely a performance and cost decision rather than a rewrite.