Study guide
1.4 How AutoML Facilitates Model and Feature Selection
Databricks Certified Machine Learning Associate Study Guide Exam
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Databricks Machine Learning (38% of the Exam)
1.1 MLOps Best Practices on Databricks 1.2 Advantages of Databricks ML Runtimes and Compute Selection 1.3 Promoting Code vs. Promoting Models 1.4 How AutoML Facilitates Model and Feature Selection 1.5 AutoML Advantages in the Model Development Process 1.6 Feature Tables in Unity Catalog vs. the Workspace Feature Store 1.7 Creating a Feature Table and Writing Data to It 1.8 Training a Model with Features from a Feature Table 1.9 Scoring a Model with Features from a Feature Table 1.10 Online vs. Offline Feature Tables 1.11 Manually Logging Metrics, Artifacts, and Models in an MLflow Run 1.12 Identifying the Best Run with the MLflow Client API 1.13 Information Available in the MLflow UI 1.14 Registering a Model in the Unity Catalog Registry 1.15 Model Tags, Aliases, and Champion/Challenger Promotion
2 Chapter 2: Data Processing and ML Workflows (19% of the Exam)
2.1 Computing Summary Statistics on a Spark DataFrame 2.2 Creating Visualizations for Categorical and Continuous Features 2.3 Comparing Two Categorical or Two Continuous Features 2.4 Removing Outliers with Standard Deviation and IQR 2.5 When Log-Scale Transformation Is Appropriate 2.6 Comparing and Applying Mean, Median, and Mode Imputation 2.7 One-Hot Encoding and When It Is or Is Not Appropriate
3 Chapter 3: Model Development (31% of the Exam)
3.1 Selecting the Appropriate Algorithm for a Scenario 3.2 Methods to Mitigate Data Imbalance in Training Data 3.3 Comparing Estimators and Transformers 3.4 Developing a Training Pipeline 3.5 Tuning Hyperparameters with Hyperopt's fmin 3.6 Grid, Random, and Bayesian Search Strategies 3.7 Parallelizing Single-Node Models for Hyperparameter Tuning 3.8 Cross-Validation vs. a Train-Validation Split 3.9 Counting the Models Trained by Grid Search and Cross-Validation 3.10 Using Common Classification Metrics 3.11 Using Common Regression Metrics 3.12 Choosing the Most Appropriate Metric for the Objective 3.13 Exponentiating Log-Transformed Predictions Before Evaluation 3.14 Model Complexity and the Bias-Variance Tradeoff
4 Chapter 4: Model Deployment (12% of the Exam)
4.1 Differences and Advantages of Batch, Streaming, and Real-Time Serving 4.2 Performing Batch Inference with pandas and Spark UDFs 4.3 Streaming Inference with Delta Live Tables 4.4 Deploying a Custom Model to a Model Endpoint 4.5 Deploying, Querying, and Splitting Traffic Across Real-Time Endpoints
1.4 How AutoML Facilitates Model and Feature Selection
Key Takeaways Explain This Topic AI
Databricks AutoML performs model selection by training candidates from several families — LightGBM, XGBoost, random forest, decision tree, and linear or logistic regression — and ranking them on one primary metric.
Feature selection is handled implicitly: AutoML detects semantic column types, drops unsupported and constant columns, truncates high-cardinality categoricals, and honours an explicit exclude_cols list.
Every trial logs a SHAP feature-importance plot, so AutoML doubles as a fast feature-signal survey before any manual engineering work begins.
primary_metric defaults to f1 for classification, r2 for regression, and smape for forecasting, and it is the single value used to rank the leaderboard.
AutoML deliberately does not invent domain features — ratios, windowed aggregates, and interaction terms remain the data scientist's job, typically materialised in a Unity Catalog feature table first.
Last updated: August 2026
- 1.4 How AutoML Facilitates Model and Feature Selection
- Databricks AutoML answers two questions that otherwise consume the first week of
- any tabular project: which algorithm family fits this data , and which columns
- actually carry signal . It answers both empirically — by training and ranking real
candidate models — rather than by heuristic.
Supported Problem Types and Algorithms
Databricks AutoML supports three fundamental supervised learning paradigms:
Classification (Binary and Multi-Class) :
Objective : Predict discrete categorical labels (e.g., customer churn: 0 or 1 , fraud classification, support ticket categorization).
Candidate Model Families : LightGBM, XGBoost, Random Forest, Decision Tree, Logistic Regression.
Primary Metric : f1 is the default; log_loss , precision , accuracy , and roc_auc can be selected with primary_metric .
Regression (Continuous Numeric Prediction) :
Objective : Predict real-valued continuous quantities (e.g., housing prices, lifetime customer value, transaction volume).
Candidate Model Families : LightGBM Regressor, XGBoost Regressor, Random Forest Regressor, Decision Tree Regressor, Linear Regression (Ridge / Lasso / ElasticNet).
Primary Metric : r2 is the default; rmse , mae , and mse can be selected with primary_metric .
Forecasting (Time-Series Extrapolation) :
Objective : Predict future sequential values across single or multiple time-series entities (e.g., store-item retail sales forecasting).
Candidate Model Families : Facebook Prophet, AutoARIMA.
Primary Metric : smape is the default; mse , rmse , mae , and mdape can be selected with primary_metric .
Automated Data Preprocessing & Validation Pipeline
Before initiating model training, Databricks AutoML performs automated sanity checks and feature transformations on the input dataset:
Transformation Stage Automated Operation Applied Rationale / Exam Relevance
Data Type Detection Infers numeric, categorical, datetime, text, and unsupported data types. Columns containing images, complex JSON arrays, or unstructured binary objects are automatically flagged and excluded.
Missing Value Imputation - Numeric features: Imputed using median (robust to outliers) or mean.; - Categorical features: Imputed with a constant "missing" token or mode. Tree models handle missing values natively, but linear models and preprocessing pipelines require clean arrays.
Categorical Encoding - Low-cardinality categories: One-Hot Encoded ( OneHotEncoder ).; - High-cardinality categories: Top categories retained, remaining grouped or hashed. Prevents exponential feature dimension explosion and sparse matrix memory exhaustion.
Extreme Values & Outliers Clamps or handles out-of-range numeric values based on feature distributions. Prevents gradient explosion in linear and neural baseline algorithms.
Dataset Splitting - Classification: Stratified Train/Val/Test split (e.g., 60/20/20).; - Regression: Random uniform split.; - Forecasting: Chronological/temporal split (training on past, validating on future). Guarantees unbiased validation without lookahead leakage.
Exam Trap Alert : Databricks AutoML does not perform heavy feature engineering such as domain-specific ratios, polynomial interaction features, or window aggregations. It focuses strictly on standard cleaning, imputation, and encoding. Custom domain features should be engineered in Unity Catalog Feature Tables prior to feeding data into AutoML.
- How AutoML Narrows the Feature Set
- AutoML does not run a formal feature-selection algorithm such as recursive feature
- elimination. It reduces the feature space through a sequence of concrete, inspectable
decisions:
- Semantic type detection. Each column is classified as numeric, categorical,
- datetime, or text. Columns AutoML cannot represent — images, nested arrays, binary
blobs — are dropped and reported in the data exploration notebook.
Uninformative column removal. Constant columns and columns that are null in
effectively every row carry no signal and are excluded before training.
- High-cardinality truncation. A categorical column with thousands of levels
- would explode the one-hot matrix, so AutoML retains the most frequent levels and
consolidates the tail rather than encoding every value.
- Explicit exclusion. exclude_cols is how you remove leakage: primary keys,
- post-outcome timestamps, or any column computed after the label was known. This
is the parameter exam scenarios reach for when a model scores suspiciously well.
- Post-hoc importance, on request. Each generated trial notebook contains SHAP
- code, but it is gated behind shap_enabled = False , because the computation is
- memory-intensive. Setting shap_enabled = True in the notebook and re-running the
- feature-importance cell produces the summary plot that tells you which features the
- model actually relied on — the practical feature-selection output of an AutoML run,
and one you have to opt into.
- Bringing curated features in
- feature_store_lookups lets AutoML join Unity Catalog feature tables into the
- training set by lookup key, with an optional timestamp key for point-in-time
- correctness. That keeps engineered features governed and reusable instead of
recomputed inside a one-off notebook.
- Exam trap : AutoML performs cleaning and encoding , not feature creation . It
- will impute, encode, and split; it will not build a 30-day rolling spend ratio or a
- customer-tenure interaction term. Domain features belong in a feature table before
AutoML runs — see Section 1.7.
Reading the leaderboard as a feature-selection tool
AutoML's output is more than a winning model, and two artefacts drive feature decisions.
- The data exploration notebook is generated automatically and imported into the
- workspace alongside the best trial notebook. It profiles the input data and raises
- warnings for high cardinality, strong correlations, and null-heavy columns, which is
usually the fastest read on which columns are worth keeping.
- Feature importance comes from the trial notebooks, and it is opt-in rather than
- automatic: the SHAP cell ships with shap_enabled = False and has to be switched on and
- re-run. Doing that for two or three notebooks near the top of the leaderboard is worth
- the compute, because agreement across different algorithms is stronger evidence than any
- single model's ranking — when a linear model and a tree ensemble independently put the
- same three columns first, that is the justification for pruning a wide feature set before
- a hand-built pipeline is written. Note that only the data exploration notebook and the
- best trial notebook land in the workspace automatically; the remaining trial notebooks
are stored as MLflow artifacts and are imported on demand from the experiment UI or with databricks.automl.import_notebook .