ExamFinch.com My Study

Study guide

3.3 Views: Temporary vs Global Temporary Views

Practice this exam

Databricks Certified Associate Developer for Apache Spark — Free Study Guide 2026 Exam

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Spark Architecture and Execution

1.1 Spark Benefits, Modules, and DataFrame/Dataset Concepts 1.2 Driver, Executors, and Cluster Managers 1.3 Execution Hierarchy: Applications, Jobs, Stages, and Tasks 1.4 Lazy Evaluation, Transformations vs Actions, and Resilient Lineage

2 Chapter 2: Shuffles, Memory and Persistence

2.1 Narrow vs Wide Dependencies and the Shuffle Mechanism 2.2 Memory Management, Fault Tolerance, and Garbage Collection 2.3 Caching, Persistence, and StorageLevels

3 Chapter 3: Spark SQL Sessions, Tables and Views

3.1 SparkSession, Catalogs, Databases, and Namespaces 3.2 Managed vs External Tables and CTAS 3.3 Views: Temporary vs Global Temporary Views

4 Chapter 4: SQL Sources and Stored Layout

4.1 JDBC Reads, Writes, and Parallelism 4.2 Direct File SQL, Save Modes, and Persistent Layout

5 Chapter 5: SQL Queries and Analytics

5.1 SQL Queries, CTEs, Subqueries, and Built-in Expressions 5.2 Built-in SQL Expressions and an Integrated Query 5.3 Window Functions and Complex Joins in Spark SQL 5.4 SQL Join Semantics and Integrated Analytics

6 Chapter 6: DataFrame Construction and Columns

6.1 DataFrame Construction, Schemas, and StructTypes 6.2 Column Operations: Selecting, Renaming, Deriving, and Casting 6.3 Complex Nested Types and User-Defined Functions

7 Chapter 7: DataFrame Validation and Inspection

7.1 Row Operations: Filtering, Sorting, and Deduplication 7.2 Handling Missing and Null Data 7.3 Schema Inspection, Row Iteration, and List Conversion

8 Chapter 8: Aggregations and Dates

8.1 Grouping, Multi-Dimensional Aggregations, and Pivoting 8.2 Dates, Unix Epoch, and Time-Zone Conversions

9 Chapter 9: DataFrame Joins and File IO

9.1 DataFrame Joins and Combining Datasets 9.2 Data Sources, SaveModes, and Partitioning

10 Chapter 10: Shared Variables and Stateful Processing

10.1 Broadcast Variables, Accumulators, and Closures 10.2 Stateless UDFs, Stateful Operators, and StateStores

11 Chapter 11: Troubleshooting and Tuning

11.1 Diagnosing Performance with the Spark UI & Execution Plans 11.2 Data Skew and Partition Tuning: repartition vs coalesce 11.3 Join Optimization and Broadcast Hash Joins

12 Chapter 12: Structured Streaming

12.1 Streaming Architecture, Sources, and Sinks 12.2 Output Modes, Triggers, and Checkpointing 12.3 Event-Time Processing, Watermarking, and Windowing 12.4 Streaming Deduplication With and Without Watermarks

13 Chapter 13: Spark Connect and Deployment

13.1 Spark Connect Architecture & Client-Server Decoupling 13.2 Working with Spark Connect: Connections and Best Practices

14 Chapter 14: Pandas API on Spark

14.1 Pandas API on Apache Spark: Core Architecture and Syntax 14.2 Pandas-on-Spark Operations, Indexing, and Optimization

3.3 Views: Temporary vs Global Temporary Views

Key Takeaways Explain This Topic AI

Relational views in Spark SQL represent logical execution plans rather than materialized physical tables; querying a view incorporates its query plan directly into Catalyst optimization.

Standard temporary views created via createTempView() or createOrReplaceTempView() are strictly session-scoped and are dropped automatically when the creating SparkSession terminates.

Global temporary views created via createGlobalTempView() or createOrReplaceGlobalTempView() are application-scoped, shared across all SparkSessions within the same driver JVM.

Global temporary views are maintained within a dedicated system database named global_temp ; all queries referencing them MUST qualify the view identifier with global_temp.<view_name> .

Last updated: October 2026

In Apache Spark SQL, views allow developers to register DataFrames or SQL transformation logic as queryable relational objects without incurring the storage overhead of writing physical files to cloud storage.

1. Relational Views in Apache Spark

A view is a virtual table defined by a logical query plan. Unlike physical tables:

Physical Tables persist both metadata (in the catalog metastore) and physical data files (Parquet, Delta, ORC) on storage.

Views store only the query definition. When a view is queried, Spark embeds the view's logical plan directly into the outer query's plan tree. The Catalyst optimizer can then perform cross-boundary optimizations, including predicate pushdown, projection pruning, and join reordering.

Spark SQL supports three distinct view varieties:

  • Standard Temporary Views (Session-scoped)
  • Global Temporary Views (Application-scoped)
  • Permanent Catalog Views (Metastore-scoped)
  • 2. Standard Temporary Views (Session-Scoped)

A standard temporary view is tied strictly to the lifecycle of the specific SparkSession that created it. It is not registered in an external metastore (Hive or Unity Catalog) and is stored entirely in driver memory.

  • Registration Methods in PySpark # Read a source dataset
  • orders_df = spark.read.parquet("s3://lakehouse/orders/")
  • # Create a temporary view; throws AnalysisException if 'temp_orders' already exists
  • orders_df.createTempView("temp_orders")
  • # Create or replace an existing temporary view safely
  • orders_df.createOrReplaceTempView("temp_orders")
  • Registration via SQL DDL CREATE OR REPLACE TEMPORARY VIEW temp_orders AS
  • SELECT order_id, customer_id, total_amount, order_date
  • FROM bronze_orders
  • WHERE order_status = 'COMPLETED';
  • Scope and Lifecycle Boundaries

Session Bound : Standard temporary views are accessible only within the creating SparkSession . If another session is instantiated using spark2 = spark.newSession() , executing spark2.table("temp_orders") throws an AnalysisException: Table or view not found .

Automatic Termination : When the SparkSession is stopped via spark.stop() or the notebook/script session finishes, the temporary view is immediately destroyed and deallocated from driver memory.

  • Programmatic Dropping
  • To drop a standard temporary view before session termination: # Programmatic drop via Catalog API
  • spark.catalog.dropTempView("temp_orders")
  • In SQL: DROP VIEW IF EXISTS temp_orders;
  • 3. Global Temporary Views (Application-Scoped)

When multiple SparkSession instances running within the same Spark application (the same Driver JVM) must collaborate on a shared in-memory relation, a standard temporary view is insufficient. Spark provides global temporary views to satisfy this requirement.

  • Registration Methods in PySpark # Create an application-wide global temporary view
  • orders_df.createGlobalTempView("global_orders") # Throws AnalysisException if exists
  • # Create or replace safely
  • orders_df.createOrReplaceGlobalTempView("global_orders")
  • Registration via SQL DDL CREATE OR REPLACE GLOBAL TEMPORARY VIEW global_orders AS
  • SELECT order_id, customer_id, total_amount
  • FROM bronze_orders
  • WHERE total_amount > 100.00;
  • The System-Managed global_temp Database

To prevent naming collisions between session-scoped temporary views and application-wide global temporary views, Spark automatically places all global temporary views into a dedicated system-preserved database named global_temp .

Mandatory Namespace Qualification

Whenever a query references a global temporary view, the developer must explicitly qualify the view name with the global_temp database prefix: # CORRECT: Fully qualified with global_temp

  • spark.sql("SELECT * FROM global_temp.global_orders").show()
  • df_global = spark.table("global_temp.global_orders")
  • # INCORRECT: Fails with AnalysisException: Table or view not found: global_orders
  • # spark.sql("SELECT * FROM global_orders").show()

This is one of the most frequently tested topics on Databricks developer examinations. Unqualified access fails because Spark searches the current active database and the local session's temporary views, ignoring the global_temp database unless explicitly instructed.

Lifecycle Boundaries

Application Bound : Global temporary views remain alive as long as the Spark application driver is running.

Cross-Session Visibility : Any session created within the driver JVM (via spark.newSession() ) can query global_temp.global_orders .

Explicit Dropping : A global temporary view can be dropped using spark.catalog.dropGlobalTempView("global_orders") or SQL DROP VIEW IF EXISTS global_temp.global_orders; .

4. Multi-Session Code Walkthrough

The following complete PySpark example demonstrates the isolation and sharing boundaries between standard temporary views and global temporary views across sessions: from pyspark.sql import SparkSession

  • # 1. Initialize primary SparkSession
  • spark = SparkSession.builder.appName("ViewScopingWalkthrough").getOrCreate()
  • # 2. Construct sample DataFrame
  • data = [(1, "Alice", 500), (2, "Bob", 1200), (3, "Charlie", 800)]
  • columns = ["id", "name", "score"]
  • df = spark.createDataFrame(data, columns)
  • # 3. Register standard and global temporary views
  • df.createOrReplaceTempView("local_view")
  • df.createOrReplaceGlobalTempView("shared_view")
  • # 4. Query within the creating session
  • spark.sql("SELECT * FROM local_view").show() # SUCCEEDS
  • spark.sql("SELECT * FROM global_temp.shared_view").show() # SUCCEEDS
  • # 5. Spawn an isolated child session
  • child_session = spark.newSession()

# 6. Attempt to query local_view from child_session

try:

child_session.sql("SELECT * FROM local_view").show()

except Exception as e:

  • print("child_session cannot access local_view:", type(e).__name__)
  • # Output: child_session cannot access local_view: AnalysisException
  • # 7. Query shared_view from child_session
  • child_session.sql("SELECT * FROM global_temp.shared_view").show()
  • # SUCCEEDS: Global temporary views are visible across all sessions
  • 5. Architectural Comparison: Views vs Tables

Feature Dimension Standard Temporary View Global Temporary View Permanent Catalog View Managed / External Table

Lifecycle Scope Current SparkSession Whole Spark Application (Driver JVM) Metastore lifetime (persistent) Metastore lifetime (persistent)

Storage Footprint None (in-memory plan) None (in-memory plan) None (in-memory plan) Physical data files on disk/cloud

Metastore Registration No (driver memory only) No (driver memory only) Yes (catalog metastore) Yes (catalog metastore)

  • Cross-Session Visible? No Yes Yes Yes
  • Cross-Application Visible? No No Yes Yes

Namespace Qualification Unqualified ( my_view ) Qualified ( global_temp.my_view ) Qualified or default ( db.my_view ) Qualified or default ( db.my_tbl )

Creation Method (PySpark) createOrReplaceTempView() createOrReplaceGlobalTempView() SQL CREATE VIEW saveAsTable()

Drop Method spark.catalog.dropTempView() spark.catalog.dropGlobalTempView() DROP VIEW db.my_view DROP TABLE db.my_tbl

6. Critical Exam Traps and Best Practices

When preparing for the certification exam, be vigilant regarding these subtle behaviors:

Omitting the global_temp. Prefix : Never write SELECT * FROM my_global_view . The global temporary database is reserved and cannot be selected as an ordinary current database; use global_temp.<view_name>.

createTempView vs createOrReplaceTempView : Calling df.createTempView("v") when v already exists throws an AnalysisException: Temporary table 'v' already exists . Always use createOrReplaceTempView when idempotent view registration is required.

View Plan Re-evaluation : Because views store query plans rather than cached data, querying a view three times recalculates the underlying transformations three times. If a view encapsulates an expensive shuffle or aggregations, call spark.catalog.cacheTable("view_name") or df.cache() to prevent redundant cluster computation.