ExamFinch.com My Study

Study guide

1.12 Identifying the Best Run with the MLflow Client API

Practice this exam

Databricks Certified Machine Learning Associate Study Guide Exam

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Databricks Machine Learning (38% of the Exam)

1.1 MLOps Best Practices on Databricks 1.2 Advantages of Databricks ML Runtimes and Compute Selection 1.3 Promoting Code vs. Promoting Models 1.4 How AutoML Facilitates Model and Feature Selection 1.5 AutoML Advantages in the Model Development Process 1.6 Feature Tables in Unity Catalog vs. the Workspace Feature Store 1.7 Creating a Feature Table and Writing Data to It 1.8 Training a Model with Features from a Feature Table 1.9 Scoring a Model with Features from a Feature Table 1.10 Online vs. Offline Feature Tables 1.11 Manually Logging Metrics, Artifacts, and Models in an MLflow Run 1.12 Identifying the Best Run with the MLflow Client API 1.13 Information Available in the MLflow UI 1.14 Registering a Model in the Unity Catalog Registry 1.15 Model Tags, Aliases, and Champion/Challenger Promotion

2 Chapter 2: Data Processing and ML Workflows (19% of the Exam)

2.1 Computing Summary Statistics on a Spark DataFrame 2.2 Creating Visualizations for Categorical and Continuous Features 2.3 Comparing Two Categorical or Two Continuous Features 2.4 Removing Outliers with Standard Deviation and IQR 2.5 When Log-Scale Transformation Is Appropriate 2.6 Comparing and Applying Mean, Median, and Mode Imputation 2.7 One-Hot Encoding and When It Is or Is Not Appropriate

3 Chapter 3: Model Development (31% of the Exam)

3.1 Selecting the Appropriate Algorithm for a Scenario 3.2 Methods to Mitigate Data Imbalance in Training Data 3.3 Comparing Estimators and Transformers 3.4 Developing a Training Pipeline 3.5 Tuning Hyperparameters with Hyperopt's fmin 3.6 Grid, Random, and Bayesian Search Strategies 3.7 Parallelizing Single-Node Models for Hyperparameter Tuning 3.8 Cross-Validation vs. a Train-Validation Split 3.9 Counting the Models Trained by Grid Search and Cross-Validation 3.10 Using Common Classification Metrics 3.11 Using Common Regression Metrics 3.12 Choosing the Most Appropriate Metric for the Objective 3.13 Exponentiating Log-Transformed Predictions Before Evaluation 3.14 Model Complexity and the Bias-Variance Tradeoff

4 Chapter 4: Model Deployment (12% of the Exam)

4.1 Differences and Advantages of Batch, Streaming, and Real-Time Serving 4.2 Performing Batch Inference with pandas and Spark UDFs 4.3 Streaming Inference with Delta Live Tables 4.4 Deploying a Custom Model to a Model Endpoint 4.5 Deploying, Querying, and Splitting Traffic Across Real-Time Endpoints

1.12 Identifying the Best Run with the MLflow Client API

Key Takeaways Explain This Topic AI

mlflow.search_runs() returns a pandas DataFrame of runs; MlflowClient().search_runs() returns Run objects and is the API used inside pipelines.

Ranking is done with order_by=["metrics.val_f1 DESC"] plus max_results=1 ; do not sort a collected DataFrame when the experiment is large.

Filter strings are SQL-like and require the entity prefix: metrics. , params. , tags. , or attributes. — and parameter values are compared as strings.

run_view_type=ViewType.ACTIVE_ONLY (the default) excludes deleted runs; ALL includes them, which is a common source of surprising 'best' runs.

The best run's run_id yields the model URI runs:/<run_id>/<artifact_path> , which is what gets passed to mlflow.register_model .

Last updated: August 2026

  • 1.12 Identifying the Best Run with the MLflow Client API
  • After a sweep of dozens or hundreds of trials, the best run has to be identified by
  • code, not by eye — a promotion job cannot scroll the UI. MLflow exposes two search

entry points, and the exam expects you to know both the syntax and the difference.

  • Two APIs, Two Return Types import mlflow
  • from mlflow.tracking import MlflowClient
  • from mlflow.entities import ViewType

experiment = mlflow.get_experiment_by_name("/Shared/Experiments/churn_prediction")

  • # A) Fluent API -> pandas DataFrame, convenient for analysis in a notebook
  • runs_df = mlflow.search_runs(
  • experiment_ids=[experiment.experiment_id],
  • filter_string="metrics.val_f1 > 0.80 AND params.model_family = 'xgboost'",
  • order_by=["metrics.val_f1 DESC"],
  • max_results=1,
  • )
  • best_run_id = runs_df.iloc[0]["run_id"]
  • # B) Client API -> list of Run objects, preferred inside jobs and pipelines
  • client = MlflowClient()
  • best = client.search_runs(
  • experiment_ids=[experiment.experiment_id],
  • filter_string="metrics.val_f1 > 0.80",
  • run_view_type=ViewType.ACTIVE_ONLY,
  • order_by=["metrics.val_f1 DESC"],
  • max_results=1,
  • )[0]
  • print(best.info.run_id, best.data.metrics["val_f1"], best.data.params["max_depth"])
  • Use the fluent version when you want a DataFrame to plot or aggregate. Use the client

version in automation: it avoids materialising a DataFrame, exposes the full Run entity (info, data, inputs), and is the same object the rest of the client API

consumes.

Filter-String Grammar

Every term must be prefixed with the entity it refers to:

  • Prefix Refers to Example
  • metrics. A logged metric (numeric comparison) metrics.val_auc >= 0.9
  • params. A logged parameter ( string comparison) params.max_depth = '6'
  • tags. A run tag tags.mlflow.runName LIKE 'sweep%'
  • attributes. Run attributes attributes.status = 'FINISHED'

Rules that trip people up:

Parameters are stored as strings, so the comparison value must be quoted: params.max_depth = '6' , never params.max_depth = 6 .

Conjunction is AND in upper case. OR is not supported in MLflow filter

strings — a filter that needs a disjunction must be split into two searches.

Metric comparisons use the metric's latest logged value for that key.

Comparators by Field Type

  • Field type Supported comparators
  • Numeric — metrics and numeric attributes = , != , > , >= , < , <=
  • String — params, tags, string attributes = , != , LIKE (case-sensitive), ILIKE (case-insensitive)
  • Existence — tags and params only IS NULL , IS NOT NULL

Sets — attributes.run_id and datasets.* only IN , with single-quoted members # Sweep runs that recorded an environment tag and cleared the F1 bar

  • client.search_runs(
  • experiment_ids=[experiment.experiment_id],
  • filter_string=(
  • "tags.mlflow.runName ILIKE 'sweep%' "
  • "AND tags.environment IS NOT NULL "
  • "AND metrics.val_f1 > 0.8"
  • ),
  • order_by=["metrics.val_f1 DESC"],
  • )

Two more grammar rules the exam can lean on:

  • Backtick awkward key names. A key containing a hyphen or starting with a digit
  • must be wrapped: metrics.`cross-entropy-loss` < 0.5 . Written bare, it is a

parse error rather than an empty result.

IN is narrow. It applies to attributes.run_id and the datasets.* fields

only, and its members take single quotes. There is no IN over an arbitrary param.

  • Passing search_all_experiments=True to mlflow.search_runs searches every
  • experiment the caller can read, which is convenient for a one-off audit and expensive

as a habit — prefer explicit experiment_ids in scheduled jobs.

Ordering, Limits, and View Types

order_by=["metrics.val_f1 DESC"] sorts server-side. Combined with max_results=1 , only one row crosses the wire — the correct pattern for an

experiment with thousands of runs.

Multiple keys break ties: order_by=["metrics.val_f1 DESC", "attributes.start_time ASC"] prefers the earliest run among equal scores, which makes promotion deterministic.

  • run_view_type defaults to ACTIVE_ONLY . Passing ViewType.ALL includes deleted
  • runs, and ViewType.DELETED_ONLY returns only them. If a search keeps returning a

run someone deleted, this parameter is why.

  • From Best Run to Registered Model model_uri = f"runs:/{best.info.run_id}/model"
  • version = mlflow.register_model(model_uri=model_uri,
  • name="prod_catalog.ml.churn_classifier")
  • runs:/<run_id>/<artifact_path> is the URI form for an artifact that has not been

registered yet. Once registered it is addressed as models:/catalog.schema.name/<version> or by alias — see Sections 1.14 and 1.15.

  • Finding the Trials of One Sweep
  • Hyperopt and AutoML write each trial as a child run whose mlflow.parentRunId tag
  • holds the parent's run ID. A search across the experiment returns parents and children
  • together, so ranking purely by the trial metric can surface a parent run whose metric
  • was aggregated rather than measured. Constrain the search to one sweep: children = client.search_runs(
  • experiment_ids=[experiment.experiment_id],
  • filter_string=f"tags.mlflow.parentRunId = '{parent_run_id}'",
  • order_by=["metrics.val_f1 DESC"],
  • max_results=1,
  • )
  • Exam tip : when a nested Hyperopt sweep is involved, remember that the parent run

holds the summary while each trial is a child run. Searching for the best trial means searching the experiment's runs and ordering by the trial metric, not reading

the parent.

  • Ordering, Ties, and Pagination
  • order_by accepts several keys and applies them in sequence, which is how ties are
  • broken deterministically: runs = client.search_runs(
  • experiment_ids=[experiment.experiment_id],
  • filter_string="attributes.status = 'FINISHED'",
  • order_by=["metrics.val_f1 DESC", "attributes.start_time ASC"],
  • max_results=1,
  • )
  • Without a tiebreaker, two runs holding identical metrics come back in an unspecified
  • order, so re-running the same selection logic can register a different model version

from one day to the next. Two further details matter in practice. search_runs paginates, so a result set larger than the page size requires following the returned

  • page token rather than assuming everything arrived in one call. And failed or killed
  • runs still appear in the results unless the filter excludes them — a run that crashed
  • after logging one epoch's metric can otherwise win a DESC ordering outright, which is
  • why attributes.status = 'FINISHED' belongs in the filter whenever a sweep might have

partial runs in it.