Study guide
19.1 Configuring Model Monitoring on Agent Platform
Google Cloud Professional Machine Learning Engineer PMLE
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Exam Overview & the Agent Platform Map
1.1 PMLE Exam Facts, Policies & the June 2026 Blueprint 1.2 From Vertex AI to Gemini Enterprise Agent Platform: The Product Name Map
2 Chapter 2: Low-Code ML with BigQuery ML
2.1 Choosing, Training & Predicting with BigQuery ML Models 2.2 Feature Engineering & Feature Selection in BigQuery ML 2.3 Fine-Tuning & Evaluating Gemini Models from BigQuery
3 Chapter 3: AutoML & Google Cloud AI APIs
3.1 Training Models with Agent Platform AutoML 3.2 Building with Document AI, Vision, Translation & Other AI APIs
4 Chapter 4: Foundation Models in Model Garden
4.1 Evaluating & Selecting Models in Model Garden 4.2 Building & Tuning Solutions with Gemini, Imagen, Veo & MaaS 4.3 Optimizing Gemini Applications for Cost, Latency & Availability
5 Chapter 5: Exploring, Preprocessing & Protecting Data
5.1 Organizing & Exploring Tabular, Text & Image Data 5.2 Choosing a Preprocessing Tool: BigQuery, Dataflow, Spark or Python 5.3 Creating & Consolidating Features in Agent Platform Feature Store 5.4 Protecting PII & Sensitive Data in ML Workflows
6 Chapter 6: Notebooks, Prototyping & Experiment Tracking
6.1 Workbench vs. Colab Enterprise: Secure, Collaborative Notebooks 6.2 Prototyping with PyTorch, scikit-learn, JAX & Model Garden in Notebooks 6.3 Experiment Environments, Tracking & Lineage with Experiments and ML Metadata
7 Chapter 7: Evaluating Predictive & Generative AI Solutions
7.1 Evaluation Metrics for Predictive Models 7.2 Evaluating Gen AI: Computation Metrics, Rubrics & LLM-as-a-Judge
8 Chapter 8: Choosing the Modeling Approach
8.1 Choosing the Model Type: ARIMA, Trees, DNNs & LLMs 8.2 Choosing the Product & Deployment Strategy 8.3 Modeling Techniques for Interpretability Requirements
9 Chapter 9: Training Data & Training SDKs
9.1 Organizing & Ingesting Training Data from Cloud Storage & BigQuery 9.2 Custom Training on Agent Platform: Containers, Jobs & Worker Pools 9.3 Kubeflow on GKE, AutoML & Tabular Workflows as Training Choices
10 Chapter 10: Troubleshooting, Hyperparameter Tuning & Fine-Tuning
10.1 Troubleshooting ML Training Failures 10.2 Hyperparameter Tuning on Agent Platform 10.3 Fine-Tuning Foundation Models & When to Tune
11 Chapter 11: Training Hardware & Distributed Training
11.1 Choosing CPUs, GPUs & TPUs for Training 11.2 Distributed Training: Data & Model Parallelism
12 Chapter 12: Deploying Models for Inference
12.1 Batch vs. Online Inference on Agent Platform 12.2 Serving Beyond Agent Platform Endpoints: Model Garden, Cloud Run & GKE 12.3 Prebuilt & Custom Serving Containers
13 Chapter 13: Model Versions, Rollouts & Inference Logic
13.1 Organizing & Versioning Models in Model Registry 13.2 Rollout Strategies: A/B Tests, Canaries & Traffic Splits 13.3 Inference Preprocessing & Postprocessing
14 Chapter 14: Scaling Online Serving
14.1 Serving Features Online with Feature Store 14.2 Public, Dedicated & Private Endpoints 14.3 Serving Hardware & Autoscaling for Throughput 14.4 Tuning Models for Production Training & Serving
15 Chapter 15: End-to-End ML Pipelines
15.1 Validating Data & Models in Pipelines 15.2 Agent Platform Pipelines with KFP: Components, Templates & Caching 15.3 Ensuring Consistent Preprocessing Between Training & Serving
16 Chapter 16: Orchestration with Managed Airflow & Ray
16.1 Orchestrating ML Workflows with Managed Service for Apache Airflow 16.2 Ray on Agent Platform for Distributed ML Workloads
17 Chapter 17: Automating Retraining & CI/CD/CT
17.1 Choosing a Retraining Policy 17.2 CI/CD/CT for ML with Cloud Build
18 Chapter 18: Securing AI Systems, Responsible AI & Explainability
18.1 Protecting AI Systems from Data & Model Exfiltration 18.2 Defending Gen AI Apps: Safety Filters, Model Armor & Regex Guardrails 18.3 Responsible AI: Detecting & Monitoring Bias 18.4 Model Explainability on Agent Platform
19 Chapter 19: Monitoring AI Solutions
19.1 Configuring Model Monitoring on Agent Platform 19.2 Training-Serving Skew, Data Drift, Concept Drift & Attribution Drift 19.3 Monitoring, Testing & Evaluating Gen AI Solutions in Production
19.1 Configuring Model Monitoring on Agent Platform
Key Takeaways Explain This Topic AI
Model Monitoring v1 is generally available and configured on Agent Platform endpoints, while Model Monitoring v2 (preview) attaches a model monitor to a model version in Model Registry.
Model Monitoring v2 supports tabular models served on Agent Platform or elsewhere, including GKE and BigQuery, through reference models.
Model Monitoring v1 monitors every feature with a default alert threshold of 0.3 and runs every 24 hours by default, with a minimum granularity of 1 hour.
Endpoint-based monitoring needs request-response logging, so private endpoints using VPC Network Peering can't be monitored this way.
Monitoring v2 compares a target dataset with a baseline dataset using a time window and optional offset, and it can run on demand or on a schedule.
Last updated: September 2026
The exam guide lists configuring and using Model Monitoring on Gemini Enterprise Agent Platform to establish continuous evaluation metrics for production models . Monitoring catches silent failures: the endpoint is healthy and fast, but predictions are getting worse because the data changed.
Two Offerings
Aspect Model Monitoring v1 (generally available) Model Monitoring v2 (preview)
Attached to An Agent Platform endpoint (monitoring job per endpoint) A model version in Model Registry (a model monitor per version)
Where the model runs Agent Platform endpoints Agent Platform or other infrastructure (GKE, BigQuery, and others) through reference models
Model types Tabular AutoML and imported tabular custom models Tabular models
Detection Training-serving skew (needs training data) and prediction drift Input feature drift , output inference drift , feature attribution changes, against any baseline
When to choose Production-level support for an endpoint-deployed model All other cases. Google says v2 has all v1 capabilities and more
v1 users don't have to migrate. Both versions can run side by side during a transition.
Model Monitoring v1 on an Endpoint
Ensure all models on the endpoint are tabular AutoML or imported tabular custom models.
For skew detection , provide the training dataset (Cloud Storage or BigQuery) and the target field . For drift detection , training data isn't required.
For custom models, provide an analysis instance schema (the job waits in pending until it has one).
Configure:
Alert thresholds per feature. The default is 0.3 for every feature . For example, Age=0.4 alerts when Age's distribution distance exceeds 0.4.
Sampling rate of logged prediction requests (all requests by default).
Monitoring frequency: 24 hours by default , 1 hour minimum .
Notifications: email and Cloud Monitoring channels (the console supports PagerDuty, Slack, and Pub/Sub).
- Model Monitoring v2 Building Blocks
- 1. Register the model
Register it in Model Registry . For models served elsewhere, register a reference model (a placeholder name without artifacts). Reference models can't use feature attribution monitoring.
2. Create a model monitor (one per model version)
Schema: feature fields, prediction fields, and optional ground-truth fields, with data types and repeated for arrays. AutoML Tables schemas are fetched automatically.
Defaults: objectives, training dataset, output location, and notification settings. Jobs use these unless overridden.
Output: a Cloud Storage location. The monitoring service agent needs write access, or the service creates a default bucket.
3. Choose data sources
Source Notes
BigQuery table or query Needs a timestamp column ( timestamp_field ) for time windows and continuous monitoring. Partition by timestamp for cost and speed
- Cloud Storage CSV (with header) or JSONL
- Batch inference job Monitoring runs soon after the batch job completes
Endpoint request-response logging Requires logging enabled. Tables include logging_time . Private (VPC peering) endpoints aren't supported because they don't log requests
Managed dataset Tabular dataset format
Nested features aren't supported, so flatten them first. Array columns are flattened into individual values.
4. Define baseline and target
Baseline: a reference, usually the training dataset or an earlier production window.
Target: recent data to compare.
Time specification: a start and end time range , or a window plus offset . For example, the target is the last 24 hours, and the baseline is the same 24 hours one week earlier.
Larger windows reduce noisy alerts caused by small sample sizes.
5. Select objectives and thresholds
Objective Metrics
Input feature data drift Categorical: L-infinity or Jensen-Shannon divergence. Numerical: Jensen-Shannon divergence
- Output inference data drift Same metrics, applied to predictions
- Feature attribution SHAP values, tracking changes in each feature's contribution
- 6. Run and schedule
Run on-demand jobs or scheduled runs for continuous monitoring. Scheduled runs with time specifications support BigQuery and endpoint logging sources. Model Monitoring v2 itself isn't charged while in preview, but you still pay for the BigQuery, Cloud Storage, and batch inference it uses.
Monitoring When Ground Truth Arrives
Distribution monitoring is a proxy. When labels arrive, measure real performance:
Store predictions with a request ID and timestamp (endpoint logging or batch output).
Join them with ground truth in BigQuery when outcomes are known.
Run the model evaluation component on recent windows in a scheduled pipeline, and compare with the validation baseline (Chapter 7).
Alert on performance drops, sliced by key segments.
From Alerts to Action
- Alert Typical response
- Skew on key features right after launch Check preprocessing parity and feature pipelines (Chapter 15)
- Gradual drift over weeks Investigate the cause. Retrain on recent data if accuracy is affected
Prediction drift with stable inputs Possible concept change or an upstream label or business change. Check outcomes
Attribution shift for an important feature Look for broken or changed feature definitions
Route alerts to Pub/Sub so a Cloud Run function can open incidents or trigger a retraining pipeline where the policy allows (Chapter 17).