Study guide
1.7 Creating a Feature Table and Writing Data to It
Databricks Certified Machine Learning Associate Study Guide Exam
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Databricks Machine Learning (38% of the Exam)
1.1 MLOps Best Practices on Databricks 1.2 Advantages of Databricks ML Runtimes and Compute Selection 1.3 Promoting Code vs. Promoting Models 1.4 How AutoML Facilitates Model and Feature Selection 1.5 AutoML Advantages in the Model Development Process 1.6 Feature Tables in Unity Catalog vs. the Workspace Feature Store 1.7 Creating a Feature Table and Writing Data to It 1.8 Training a Model with Features from a Feature Table 1.9 Scoring a Model with Features from a Feature Table 1.10 Online vs. Offline Feature Tables 1.11 Manually Logging Metrics, Artifacts, and Models in an MLflow Run 1.12 Identifying the Best Run with the MLflow Client API 1.13 Information Available in the MLflow UI 1.14 Registering a Model in the Unity Catalog Registry 1.15 Model Tags, Aliases, and Champion/Challenger Promotion
2 Chapter 2: Data Processing and ML Workflows (19% of the Exam)
2.1 Computing Summary Statistics on a Spark DataFrame 2.2 Creating Visualizations for Categorical and Continuous Features 2.3 Comparing Two Categorical or Two Continuous Features 2.4 Removing Outliers with Standard Deviation and IQR 2.5 When Log-Scale Transformation Is Appropriate 2.6 Comparing and Applying Mean, Median, and Mode Imputation 2.7 One-Hot Encoding and When It Is or Is Not Appropriate
3 Chapter 3: Model Development (31% of the Exam)
3.1 Selecting the Appropriate Algorithm for a Scenario 3.2 Methods to Mitigate Data Imbalance in Training Data 3.3 Comparing Estimators and Transformers 3.4 Developing a Training Pipeline 3.5 Tuning Hyperparameters with Hyperopt's fmin 3.6 Grid, Random, and Bayesian Search Strategies 3.7 Parallelizing Single-Node Models for Hyperparameter Tuning 3.8 Cross-Validation vs. a Train-Validation Split 3.9 Counting the Models Trained by Grid Search and Cross-Validation 3.10 Using Common Classification Metrics 3.11 Using Common Regression Metrics 3.12 Choosing the Most Appropriate Metric for the Objective 3.13 Exponentiating Log-Transformed Predictions Before Evaluation 3.14 Model Complexity and the Bias-Variance Tradeoff
4 Chapter 4: Model Deployment (12% of the Exam)
4.1 Differences and Advantages of Batch, Streaming, and Real-Time Serving 4.2 Performing Batch Inference with pandas and Spark UDFs 4.3 Streaming Inference with Delta Live Tables 4.4 Deploying a Custom Model to a Model Endpoint 4.5 Deploying, Querying, and Splitting Traffic Across Real-Time Endpoints
1.7 Creating a Feature Table and Writing Data to It
Key Takeaways Explain This Topic AI
FeatureEngineeringClient.create_table(name, primary_keys, df=..., timeseries_column=...) creates a Delta-backed feature table under a three-level Unity Catalog name.
Primary keys are mandatory — they are what FeatureLookup joins on later, so a table without them cannot be used for training-set construction.
timeseries_column is a single column name (not a list) and is required only when the table will serve point-in-time lookups.
fe.write_table(name, df, mode='merge') upserts on the primary keys; mode='overwrite' replaces the table contents wholesale.
Supplying df at creation defines the schema and loads the initial rows; supplying schema instead creates an empty table to be populated later by write_table .
Last updated: August 2026
- 1.7 Creating a Feature Table and Writing Data to It
- Creating a feature table is a two-step act: define the table with its keys and
schema, then write feature values into it on a schedule. Both steps go through FeatureEngineeringClient , and both are directly examinable.
Initializing and Creating Feature Tables
Feature tables in Unity Catalog are standard Delta Lake tables augmented with explicit feature metadata (such as primary key constraints). Creating and writing to feature tables is managed via the FeatureEngineeringClient .
Step 1: Client Initialization and Table Creation from databricks.feature_engineering import FeatureEngineeringClient
from pyspark.sql import functions as F
- # Initialize modern Feature Engineering Client
- fe = FeatureEngineeringClient()
- # Compute feature DataFrame with business aggregation logic
- raw_transactions = spark.table("prod_catalog.source_data.transactions")
- customer_features_df = raw_transactions.groupBy("customer_id").agg(
- F.count("transaction_id").alias("total_tx_count"),
- F.avg("amount").alias("avg_transaction_amount"),
- F.max("transaction_date").alias("last_transaction_date")
- )
- # Define the full 3-level table name
- feature_table_name = "prod_catalog.ml_features.customer_spending_features"
- # Create the feature table in Unity Catalog
- fe.create_table(
- name=feature_table_name,
- primary_keys=["customer_id"],
- df=customer_features_df,
- description="Aggregated customer lifetime transaction and spending metrics",
- tags={"domain": "finance", "tier": "gold"}
- )
- Key Parameters of fe.create_table
name : Fully qualified 3-level name ( catalog.schema.table ).
primary_keys : List of column names that uniquely identify entity records (e.g., ["customer_id"] or composite keys ["user_id", "device_id"] ). Primary keys are mandatory for feature lookups.
timeseries_column : (Optional, singular string ) The single column holding the timestamp of each feature observation. Required when the table will be used for point-in-time lookups. Writing timeseries_columns=[...] is a common and incorrect guess — the parameter takes one column name, not a list.
df : Optional PySpark DataFrame used to populate the table schema and initial data.
schema : Explicit StructType schema (required if df is omitted).
Writing and Updating Feature Tables
Feature values change continuously as new batch or streaming transactions arrive. The FeatureEngineeringClient.write_table API provides controlled update semantics: # Incremental daily batch update
new_daily_features_df = compute_daily_customer_features()
- # Upsert features into Unity Catalog table
- fe.write_table(
- name="prod_catalog.ml_features.customer_spending_features",
- df=new_daily_features_df,
- mode="merge" # Performs Delta Lake MERGE on primary keys
- )
- Write Modes Supported
mode="merge" (Recommended): Performs an intelligent Delta Lake MERGE (upsert) operation using the table's defined primary_keys . If an incoming entity record already exists, its feature values are updated; if the entity is new, a new row is inserted.
mode="overwrite" : Completely truncates and replaces the existing feature table contents with the incoming DataFrame. Used for full historical recalculations.
Choosing Between the Creation Modes
Situation Call Result
The feature DataFrame is already computed create_table(name, primary_keys, df=features_df) Schema inferred from df , rows written immediately
The table must exist before any data is produced (e.g. a streaming writer fills it) create_table(name, primary_keys, schema=struct_type) Empty governed table with the declared schema
- The table already exists and new values arrived write_table(name, df, mode="merge") Upsert on the primary keys
- Feature logic changed and history must be recomputed write_table(name, df, mode="overwrite") Contents replaced
- Rules that decide exam answers
- Primary keys are not optional. They are the join keys for FeatureLookup . A
table created without them cannot participate in create_training_set .
A composite key is a list : primary_keys=["user_id", "device_id"] . The
observation DataFrame must supply every key column at lookup time.
timeseries_column takes one string. It is only needed for point-in-time
tables; a static entity table (demographics, product attributes) omits it.
- Merge is keyed, not appended. mode="merge" matches incoming rows to existing
- rows on the primary keys, updating matches and inserting new entities. It will not
- produce duplicate rows for the same key, which is exactly why it is the default for
incremental refreshes.
- Reading back is ordinary Delta. fe.read_table(name=...) returns the feature
- table as a Spark DataFrame, and spark.table(name) works too — a feature table is a
Delta table with extra metadata, not a special storage format.
Unity Catalog Governance, Search & Automated Lineage
Because feature tables in Unity Catalog are native Delta tables, enterprise security policies apply seamlessly: -- Grant read permissions to data science group
- GRANT SELECT ON TABLE prod_catalog.ml_features.customer_spending_features
- TO `data-scientists`;
- - Grant write/update permissions to automated ETL service principal
- GRANT MODIFY ON TABLE prod_catalog.ml_features.customer_spending_features
- TO `sp-feature-pipeline`;
- Automated Lineage Graph
Unity Catalog captures end-to-end lineage automatically without manual instrumentation:
Upstream Lineage : Displays raw bronze/silver Delta source tables queried to build the feature table.
Downstream Lineage : Automatically identifies which MLflow training runs, registered models, and scheduled batch inference pipelines consume features from the table. [ Bronze Raw Logs ] ---> [ Silver Cleaned Data ] ---> [ Feature Table (Unity Catalog) ]
- |
- +-------------------------------+-------------------------------+
- v v
- [ MLflow Model Training Run ] [ Batch Scoring Pipeline ]
- |
- v
- [ Registered Model in UC ]