3.5 Tuning Hyperparameters with Hyperopt's fmin

Databricks Certified Machine Learning Associate Study Guide Exam

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Databricks Machine Learning (38% of the Exam)

1.1 MLOps Best Practices on Databricks 1.2 Advantages of Databricks ML Runtimes and Compute Selection 1.3 Promoting Code vs. Promoting Models 1.4 How AutoML Facilitates Model and Feature Selection 1.5 AutoML Advantages in the Model Development Process 1.6 Feature Tables in Unity Catalog vs. the Workspace Feature Store 1.7 Creating a Feature Table and Writing Data to It 1.8 Training a Model with Features from a Feature Table 1.9 Scoring a Model with Features from a Feature Table 1.10 Online vs. Offline Feature Tables 1.11 Manually Logging Metrics, Artifacts, and Models in an MLflow Run 1.12 Identifying the Best Run with the MLflow Client API 1.13 Information Available in the MLflow UI 1.14 Registering a Model in the Unity Catalog Registry 1.15 Model Tags, Aliases, and Champion/Challenger Promotion

2 Chapter 2: Data Processing and ML Workflows (19% of the Exam)

2.1 Computing Summary Statistics on a Spark DataFrame 2.2 Creating Visualizations for Categorical and Continuous Features 2.3 Comparing Two Categorical or Two Continuous Features 2.4 Removing Outliers with Standard Deviation and IQR 2.5 When Log-Scale Transformation Is Appropriate 2.6 Comparing and Applying Mean, Median, and Mode Imputation 2.7 One-Hot Encoding and When It Is or Is Not Appropriate

3 Chapter 3: Model Development (31% of the Exam)

3.1 Selecting the Appropriate Algorithm for a Scenario 3.2 Methods to Mitigate Data Imbalance in Training Data 3.3 Comparing Estimators and Transformers 3.4 Developing a Training Pipeline 3.5 Tuning Hyperparameters with Hyperopt's fmin 3.6 Grid, Random, and Bayesian Search Strategies 3.7 Parallelizing Single-Node Models for Hyperparameter Tuning 3.8 Cross-Validation vs. a Train-Validation Split 3.9 Counting the Models Trained by Grid Search and Cross-Validation 3.10 Using Common Classification Metrics 3.11 Using Common Regression Metrics 3.12 Choosing the Most Appropriate Metric for the Objective 3.13 Exponentiating Log-Transformed Predictions Before Evaluation 3.14 Model Complexity and the Bias-Variance Tradeoff

4 Chapter 4: Model Deployment (12% of the Exam)

4.1 Differences and Advantages of Batch, Streaming, and Real-Time Serving 4.2 Performing Batch Inference with pandas and Spark UDFs 4.3 Streaming Inference with Delta Live Tables 4.4 Deploying a Custom Model to a Model Endpoint 4.5 Deploying, Querying, and Splitting Traffic Across Real-Time Endpoints

3.5 Tuning Hyperparameters with Hyperopt's fmin

Key Takeaways Explain This Topic AI

fmin(fn, space, algo, max_evals, trials) drives the search: an objective function, a search space, a search algorithm, an evaluation budget, and a trials store.

The objective takes one dictionary of sampled hyperparameters and returns either a scalar loss or {'loss': value, 'status': STATUS_OK} .

fmin always minimises , so a metric that should be maximised must be returned negated — -roc_auc — or as 1 - metric .

hp.quniform returns floats, so integer hyperparameters such as max_depth must be cast with int() inside the objective or the estimator raises a TypeError.

hp.loguniform(label, np.log(lo), np.log(hi)) takes log-space bounds and is the correct distribution for learning rates and regularisation strengths.

Last updated: August 2026

3.5 Tuning Hyperparameters with Hyperopt's fmin

fmin is Hyperopt's entry point, and its signature encodes the whole workflow: from hyperopt import fmin, tpe, hp, STATUS_OK, Trials

best = fmin(
fn=objective_function, # callable: params dict -> loss (or result dict)
space=search_space, # dict of hp.* distributions
algo=tpe.suggest, # search algorithm: tpe.suggest, rand.suggest, anneal.suggest
max_evals=50, # evaluation budget
trials=Trials(), # where results are stored (Trials or SparkTrials)
rstate=np.random.default_rng(42), # reproducibility
)

Argument Purpose Common values
fn The objective to minimise Your training-and-scoring function
space Hyperparameter distributions A dict of hp.* expressions
algo How candidates are proposed tpe.suggest (Bayesian), rand.suggest (random), anneal.suggest
max_evals Total trials to run 20–200 depending on budget
trials Result store and execution engine Trials() (sequential) or SparkTrials() (distributed — Section 3.7)
rstate Random seed np.random.default_rng(seed)

fmin returns a dictionary of the best hyperparameters found. Note the sharp edge:

for hp.choice parameters it returns the index into the option list, not the
value. hyperopt.space_eval(space, best) converts the returned dictionary back into

actual parameter values, which is what you want before retraining a final model.

Defining Hyperopt Search Spaces ( hyperopt.hp )

Hyperopt provides a comprehensive suite of stochastic distribution primitives in the hyperopt.hp module:

Expression Description Common Machine Learning Use Case

hp.choice(label, options) Returns one element from a list/tuple of categorical options. Loss functions ( ['logloss', 'hinge'] ), optimizers ( ['adam', 'sgd'] ), tree splitters.

hp.uniform(label, low, high) Uniformly distributed continuous float in [ low , high ] [\text{low}, \text{high}] [ low , high ] . Subsampling ratios ( subsample , colsample_bytree ) in [ 0.5 , 1.0 ] [0.5, 1.0] [ 0.5 , 1.0 ] .

hp.quniform(label, low, high, q) Continuous float drawn uniformly, rounded to step size q q q : round ( uniform / q ) × q \text{round}(\text{uniform}/q) \times q round ( uniform / q ) × q . Number of trees ( n_estimators ), tree depth ( max_depth ). Must cast to int in objective!

hp.loguniform(label, low, high) Value drawn such that ln ⁡ ( x ) \ln(x) ln ( x ) is uniform in [ low , high ] [\text{low}, \text{high}] [ low , high ] . Learning rate η ∈ [ 10 − 4 , 10 − 1 ] \eta \in [10^{-4}, 10^{-1}] η ∈ [ 1 0 − 4 , 1 0 − 1 ] : hp.loguniform('lr', np.log(1e-4), np.log(1e-1)) .

hp.lognormal(label, mu, sigma) Value drawn from log-normal distribution with mean μ \mu μ and std σ \sigma σ . Positive regularization penalties with skewed distribution priors.

Important

Integer Casting in hp.quniform : hp.quniform returns a floating-point number (e.g. 6.0 ). Passing floats to integer parameters like max_depth or n_estimators in Scikit-learn or XGBoost causes runtime exceptions. Always cast integer hyperparameters: int(params['max_depth']) .

Objective Function Construction & Optimization Loop

The objective function is the evaluation harness executed on every trial. It must satisfy three strict rules:

Input: Accepts a single dictionary containing candidate hyperparameters sampled from the search space.

Optimization Direction: Hyperopt strictly minimizes the objective. If optimizing a metric where higher is better (e.g., ROC-AUC, Accuracy, R 2 R^2 R 2 ), you must return the negative metric (e.g., -roc_auc ) or 1 − metric 1 - \text{metric} 1 − metric .

Return Format: Returns either a single scalar loss (float) or a dictionary containing at minimum {'loss': loss_value, 'status': STATUS_OK} . +-----------------------------------------------------------------------------+

| HYPEROPT OBJECTIVE FUNCTION CONTRACT |
| |
| def objective(params): |
| # 1. Cast integer hyperparameters |
| max_depth = int(params['max_depth']) |
| |
| # 2. Train model on training data |
| model = XGBClassifier(max_depth=max_depth, lr=params['lr'], ...) |
| model.fit(X_train, y_train) |
| |
| # 3. Evaluate validation metric |
| val_auc = roc_auc_score(y_val, model.predict_proba(X_val)[:, 1]) |
| |
| # 4. Return negated loss for minimization |
| return {'loss': -val_auc, 'status': STATUS_OK} |
+-----------------------------------------------------------------------------+
Complete Production Code Example import numpy as np
import xgboost as xgb
from sklearn.model_selection import train_test_split
from sklearn.metrics import roc_auc_score
from hyperopt import fmin, tpe, hp, STATUS_OK, Trials
import mlflow
import mlflow.xgboost

# 1. Prepare Data
pdf = spark.table("lakehouse_gold.churn_features").toPandas()
X = pdf.drop(columns=["customer_id", "churned"])
y = pdf["churned"]
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)

# 2. Define Hyperopt Search Space
search_space = {
"max_depth": hp.quniform("max_depth", 3, 10, 1),
"learning_rate": hp.loguniform("learning_rate", np.log(0.005), np.log(0.3)),
"n_estimators": hp.quniform("n_estimators", 50, 400, 25),
"subsample": hp.uniform("subsample", 0.6, 1.0),
"colsample_bytree": hp.uniform("colsample_bytree", 0.6, 1.0),
"reg_alpha": hp.loguniform("reg_alpha", np.log(1e-3), np.log(10.0)),
"reg_lambda": hp.loguniform("reg_lambda", np.log(1e-3), np.log(10.0))
}

# 3. Define Objective Function

def objective_function(params):

# Cast discrete hyperparameters to integers
params["max_depth"] = int(params["max_depth"])
params["n_estimators"] = int(params["n_estimators"])

# Train candidate model
model = xgb.XGBClassifier(
**params,
random_state=42,
eval_metric="logloss",
use_label_encoder=False
)

model.fit(X_train, y_train, eval_set=[(X_val, y_val)], verbose=False)

# Predict and evaluate validation metric
val_probs = model.predict_proba(X_val)[:, 1]
val_auc = roc_auc_score(y_val, val_probs)

# Return negated metric because fmin minimizes loss
return {
"loss": -val_auc,
"status": STATUS_OK,
"roc_auc": val_auc
}

# 4. Execute Optimization with MLflow Tracking
mlflow.xgboost.autolog(log_models=True)
trials = Trials()

with mlflow.start_run(run_name="xgboost_hyperopt_parent"):

best_hyperparameters = fmin(
fn=objective_function,
space=search_space,
algo=tpe.suggest,
max_evals=50,
trials=trials,
rstate=np.random.default_rng(42)
)

print("Optimal Hyperparameters:", best_hyperparameters)
MLflow Automatic Experiment Tracking Integration

Databricks provides seamless, automatic integration between Hyperopt and MLflow. When mlflow.autolog() is activated before calling fmin() , MLflow automatically constructs a hierarchical nested experiment run structure: +-----------------------------------------------------------------------------+

| MLFLOW NESTED RUN HIERARCHY FOR HYPEROPT |
| |
| PARENT RUN: "hyperopt_tpe_optimization" |
| +-- Best Parameters: {learning_rate: 0.038, max_depth: 6, subsample: 0.85}|
| +-- Best Validation Loss: -0.9240 (ROC-AUC: 0.9240) |
| | |
| +---> CHILD RUN 1: Trial 001 | lr=0.120 | depth=3 | loss=-0.8650 | 2.1s |
| +---> CHILD RUN 2: Trial 002 | lr=0.005 | depth=8 | loss=-0.8120 | 4.3s |
| +---> CHILD RUN 3: Trial 003 | lr=0.045 | depth=5 | loss=-0.9100 | 3.0s |
| +---> CHILD RUN 4: Trial 004 | lr=0.038 | depth=6 | loss=-0.9240 | 3.5s * |
| +---> CHILD RUN N: Trial ... |
+-----------------------------------------------------------------------------+

Each nested child run captures:

The exact hyperparameter parameters evaluated in that trial.

Training metrics, validation metrics, and epoch loss curves.

Model artifacts and the environment dependencies needed to reload them.

Trial execution start time, duration, and status.

3.5 Tuning Hyperparameters with Hyperopt's fmin

Databricks Certified Machine Learning Associate Study Guide Exam

Track your progress as you read

1 Chapter 1: Databricks Machine Learning (38% of the Exam)

2 Chapter 2: Data Processing and ML Workflows (19% of the Exam)

3 Chapter 3: Model Development (31% of the Exam)

4 Chapter 4: Model Deployment (12% of the Exam)

3.5 Tuning Hyperparameters with Hyperopt's fmin

Key Takeaways Explain This Topic AI

Last updated: August 2026

3.5 Tuning Hyperparameters with Hyperopt's fmin

Defining Hyperopt Search Spaces ( hyperopt.hp )

Expression Description Common Machine Learning Use Case

Important

Objective Function Construction & Optimization Loop

# 3. Define Objective Function

model.fit(X_train, y_train, eval_set=[(X_val, y_val)], verbose=False)

Loading diagram... Hyperopt Bayesian Optimization (TPE) Execution Cycle