Study guide

19.1 Configuring Model Monitoring on Agent Platform

Google Cloud Professional Machine Learning Engineer PMLE

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Exam Overview & the Agent Platform Map

1.1 PMLE Exam Facts, Policies & the June 2026 Blueprint 1.2 From Vertex AI to Gemini Enterprise Agent Platform: The Product Name Map

2 Chapter 2: Low-Code ML with BigQuery ML

2.1 Choosing, Training & Predicting with BigQuery ML Models 2.2 Feature Engineering & Feature Selection in BigQuery ML 2.3 Fine-Tuning & Evaluating Gemini Models from BigQuery

3 Chapter 3: AutoML & Google Cloud AI APIs

3.1 Training Models with Agent Platform AutoML 3.2 Building with Document AI, Vision, Translation & Other AI APIs

4 Chapter 4: Foundation Models in Model Garden

4.1 Evaluating & Selecting Models in Model Garden 4.2 Building & Tuning Solutions with Gemini, Imagen, Veo & MaaS 4.3 Optimizing Gemini Applications for Cost, Latency & Availability

5 Chapter 5: Exploring, Preprocessing & Protecting Data

5.1 Organizing & Exploring Tabular, Text & Image Data 5.2 Choosing a Preprocessing Tool: BigQuery, Dataflow, Spark or Python 5.3 Creating & Consolidating Features in Agent Platform Feature Store 5.4 Protecting PII & Sensitive Data in ML Workflows

6 Chapter 6: Notebooks, Prototyping & Experiment Tracking

6.1 Workbench vs. Colab Enterprise: Secure, Collaborative Notebooks 6.2 Prototyping with PyTorch, scikit-learn, JAX & Model Garden in Notebooks 6.3 Experiment Environments, Tracking & Lineage with Experiments and ML Metadata

7 Chapter 7: Evaluating Predictive & Generative AI Solutions

7.1 Evaluation Metrics for Predictive Models 7.2 Evaluating Gen AI: Computation Metrics, Rubrics & LLM-as-a-Judge

8 Chapter 8: Choosing the Modeling Approach

8.1 Choosing the Model Type: ARIMA, Trees, DNNs & LLMs 8.2 Choosing the Product & Deployment Strategy 8.3 Modeling Techniques for Interpretability Requirements

9 Chapter 9: Training Data & Training SDKs

9.1 Organizing & Ingesting Training Data from Cloud Storage & BigQuery 9.2 Custom Training on Agent Platform: Containers, Jobs & Worker Pools 9.3 Kubeflow on GKE, AutoML & Tabular Workflows as Training Choices

10 Chapter 10: Troubleshooting, Hyperparameter Tuning & Fine-Tuning

10.1 Troubleshooting ML Training Failures 10.2 Hyperparameter Tuning on Agent Platform 10.3 Fine-Tuning Foundation Models & When to Tune

11 Chapter 11: Training Hardware & Distributed Training

11.1 Choosing CPUs, GPUs & TPUs for Training 11.2 Distributed Training: Data & Model Parallelism

12 Chapter 12: Deploying Models for Inference

12.1 Batch vs. Online Inference on Agent Platform 12.2 Serving Beyond Agent Platform Endpoints: Model Garden, Cloud Run & GKE 12.3 Prebuilt & Custom Serving Containers

13 Chapter 13: Model Versions, Rollouts & Inference Logic

13.1 Organizing & Versioning Models in Model Registry 13.2 Rollout Strategies: A/B Tests, Canaries & Traffic Splits 13.3 Inference Preprocessing & Postprocessing

14 Chapter 14: Scaling Online Serving

14.1 Serving Features Online with Feature Store 14.2 Public, Dedicated & Private Endpoints 14.3 Serving Hardware & Autoscaling for Throughput 14.4 Tuning Models for Production Training & Serving

15 Chapter 15: End-to-End ML Pipelines

15.1 Validating Data & Models in Pipelines 15.2 Agent Platform Pipelines with KFP: Components, Templates & Caching 15.3 Ensuring Consistent Preprocessing Between Training & Serving

16 Chapter 16: Orchestration with Managed Airflow & Ray

16.1 Orchestrating ML Workflows with Managed Service for Apache Airflow 16.2 Ray on Agent Platform for Distributed ML Workloads

17 Chapter 17: Automating Retraining & CI/CD/CT

17.1 Choosing a Retraining Policy 17.2 CI/CD/CT for ML with Cloud Build

18 Chapter 18: Securing AI Systems, Responsible AI & Explainability

18.1 Protecting AI Systems from Data & Model Exfiltration 18.2 Defending Gen AI Apps: Safety Filters, Model Armor & Regex Guardrails 18.3 Responsible AI: Detecting & Monitoring Bias 18.4 Model Explainability on Agent Platform

19 Chapter 19: Monitoring AI Solutions

19.1 Configuring Model Monitoring on Agent Platform 19.2 Training-Serving Skew, Data Drift, Concept Drift & Attribution Drift 19.3 Monitoring, Testing & Evaluating Gen AI Solutions in Production

19.1 Configuring Model Monitoring on Agent Platform

Key Takeaways Explain This Topic AI

Model Monitoring v1 is generally available and configured on Agent Platform endpoints, while Model Monitoring v2 (preview) attaches a model monitor to a model version in Model Registry.

Model Monitoring v2 supports tabular models served on Agent Platform or elsewhere, including GKE and BigQuery, through reference models.

Model Monitoring v1 monitors every feature with a default alert threshold of 0.3 and runs every 24 hours by default, with a minimum granularity of 1 hour.

Endpoint-based monitoring needs request-response logging, so private endpoints using VPC Network Peering can't be monitored this way.

Monitoring v2 compares a target dataset with a baseline dataset using a time window and optional offset, and it can run on demand or on a schedule.

Last updated: September 2026

The exam guide lists configuring and using Model Monitoring on Gemini Enterprise Agent Platform to establish continuous evaluation metrics for production models . Monitoring catches silent failures: the endpoint is healthy and fast, but predictions are getting worse because the data changed.

Two Offerings

Aspect Model Monitoring v1 (generally available) Model Monitoring v2 (preview)

Attached to An Agent Platform endpoint (monitoring job per endpoint) A model version in Model Registry (a model monitor per version)

Where the model runs Agent Platform endpoints Agent Platform or other infrastructure (GKE, BigQuery, and others) through reference models

Model types Tabular AutoML and imported tabular custom models Tabular models

Detection Training-serving skew (needs training data) and prediction drift Input feature drift , output inference drift , feature attribution changes, against any baseline

When to choose Production-level support for an endpoint-deployed model All other cases. Google says v2 has all v1 capabilities and more

v1 users don't have to migrate. Both versions can run side by side during a transition.

Model Monitoring v1 on an Endpoint

Ensure all models on the endpoint are tabular AutoML or imported tabular custom models.

For skew detection , provide the training dataset (Cloud Storage or BigQuery) and the target field . For drift detection , training data isn't required.

For custom models, provide an analysis instance schema (the job waits in pending until it has one).

Configure:

Alert thresholds per feature. The default is 0.3 for every feature . For example, Age=0.4 alerts when Age's distribution distance exceeds 0.4.

Sampling rate of logged prediction requests (all requests by default).

Monitoring frequency: 24 hours by default , 1 hour minimum .

Notifications: email and Cloud Monitoring channels (the console supports PagerDuty, Slack, and Pub/Sub).

  • Model Monitoring v2 Building Blocks
  • 1. Register the model

Register it in Model Registry . For models served elsewhere, register a reference model (a placeholder name without artifacts). Reference models can't use feature attribution monitoring.

2. Create a model monitor (one per model version)

Schema: feature fields, prediction fields, and optional ground-truth fields, with data types and repeated for arrays. AutoML Tables schemas are fetched automatically.

Defaults: objectives, training dataset, output location, and notification settings. Jobs use these unless overridden.

Output: a Cloud Storage location. The monitoring service agent needs write access, or the service creates a default bucket.

3. Choose data sources

Source Notes

BigQuery table or query Needs a timestamp column ( timestamp_field ) for time windows and continuous monitoring. Partition by timestamp for cost and speed

  • Cloud Storage CSV (with header) or JSONL
  • Batch inference job Monitoring runs soon after the batch job completes

Endpoint request-response logging Requires logging enabled. Tables include logging_time . Private (VPC peering) endpoints aren't supported because they don't log requests

Managed dataset Tabular dataset format

Nested features aren't supported, so flatten them first. Array columns are flattened into individual values.

4. Define baseline and target

Baseline: a reference, usually the training dataset or an earlier production window.

Target: recent data to compare.

Time specification: a start and end time range , or a window plus offset . For example, the target is the last 24 hours, and the baseline is the same 24 hours one week earlier.

Larger windows reduce noisy alerts caused by small sample sizes.

5. Select objectives and thresholds

Objective Metrics

Input feature data drift Categorical: L-infinity or Jensen-Shannon divergence. Numerical: Jensen-Shannon divergence

  • Output inference data drift Same metrics, applied to predictions
  • Feature attribution SHAP values, tracking changes in each feature's contribution
  • 6. Run and schedule

Run on-demand jobs or scheduled runs for continuous monitoring. Scheduled runs with time specifications support BigQuery and endpoint logging sources. Model Monitoring v2 itself isn't charged while in preview, but you still pay for the BigQuery, Cloud Storage, and batch inference it uses.

Monitoring When Ground Truth Arrives

Distribution monitoring is a proxy. When labels arrive, measure real performance:

Store predictions with a request ID and timestamp (endpoint logging or batch output).

Join them with ground truth in BigQuery when outcomes are known.

Run the model evaluation component on recent windows in a scheduled pipeline, and compare with the validation baseline (Chapter 7).

Alert on performance drops, sliced by key segments.

From Alerts to Action

  • Alert Typical response
  • Skew on key features right after launch Check preprocessing parity and feature pipelines (Chapter 15)
  • Gradual drift over weeks Investigate the cause. Retrain on recent data if accuracy is affected

Prediction drift with stable inputs Possible concept change or an upstream label or business change. Check outcomes

Attribution shift for an important feature Look for broken or changed feature definitions

Route alerts to Pub/Sub so a Cloud Run function can open incidents or trigger a retraining pipeline where the policy allows (Chapter 17).