Study guide
17.1 Choosing a Retraining Policy
Google Cloud Professional Machine Learning Engineer PMLE
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Exam Overview & the Agent Platform Map
1.1 PMLE Exam Facts, Policies & the June 2026 Blueprint 1.2 From Vertex AI to Gemini Enterprise Agent Platform: The Product Name Map
2 Chapter 2: Low-Code ML with BigQuery ML
2.1 Choosing, Training & Predicting with BigQuery ML Models 2.2 Feature Engineering & Feature Selection in BigQuery ML 2.3 Fine-Tuning & Evaluating Gemini Models from BigQuery
3 Chapter 3: AutoML & Google Cloud AI APIs
3.1 Training Models with Agent Platform AutoML 3.2 Building with Document AI, Vision, Translation & Other AI APIs
4 Chapter 4: Foundation Models in Model Garden
4.1 Evaluating & Selecting Models in Model Garden 4.2 Building & Tuning Solutions with Gemini, Imagen, Veo & MaaS 4.3 Optimizing Gemini Applications for Cost, Latency & Availability
5 Chapter 5: Exploring, Preprocessing & Protecting Data
5.1 Organizing & Exploring Tabular, Text & Image Data 5.2 Choosing a Preprocessing Tool: BigQuery, Dataflow, Spark or Python 5.3 Creating & Consolidating Features in Agent Platform Feature Store 5.4 Protecting PII & Sensitive Data in ML Workflows
6 Chapter 6: Notebooks, Prototyping & Experiment Tracking
6.1 Workbench vs. Colab Enterprise: Secure, Collaborative Notebooks 6.2 Prototyping with PyTorch, scikit-learn, JAX & Model Garden in Notebooks 6.3 Experiment Environments, Tracking & Lineage with Experiments and ML Metadata
7 Chapter 7: Evaluating Predictive & Generative AI Solutions
7.1 Evaluation Metrics for Predictive Models 7.2 Evaluating Gen AI: Computation Metrics, Rubrics & LLM-as-a-Judge
8 Chapter 8: Choosing the Modeling Approach
8.1 Choosing the Model Type: ARIMA, Trees, DNNs & LLMs 8.2 Choosing the Product & Deployment Strategy 8.3 Modeling Techniques for Interpretability Requirements
9 Chapter 9: Training Data & Training SDKs
9.1 Organizing & Ingesting Training Data from Cloud Storage & BigQuery 9.2 Custom Training on Agent Platform: Containers, Jobs & Worker Pools 9.3 Kubeflow on GKE, AutoML & Tabular Workflows as Training Choices
10 Chapter 10: Troubleshooting, Hyperparameter Tuning & Fine-Tuning
10.1 Troubleshooting ML Training Failures 10.2 Hyperparameter Tuning on Agent Platform 10.3 Fine-Tuning Foundation Models & When to Tune
11 Chapter 11: Training Hardware & Distributed Training
11.1 Choosing CPUs, GPUs & TPUs for Training 11.2 Distributed Training: Data & Model Parallelism
12 Chapter 12: Deploying Models for Inference
12.1 Batch vs. Online Inference on Agent Platform 12.2 Serving Beyond Agent Platform Endpoints: Model Garden, Cloud Run & GKE 12.3 Prebuilt & Custom Serving Containers
13 Chapter 13: Model Versions, Rollouts & Inference Logic
13.1 Organizing & Versioning Models in Model Registry 13.2 Rollout Strategies: A/B Tests, Canaries & Traffic Splits 13.3 Inference Preprocessing & Postprocessing
14 Chapter 14: Scaling Online Serving
14.1 Serving Features Online with Feature Store 14.2 Public, Dedicated & Private Endpoints 14.3 Serving Hardware & Autoscaling for Throughput 14.4 Tuning Models for Production Training & Serving
15 Chapter 15: End-to-End ML Pipelines
15.1 Validating Data & Models in Pipelines 15.2 Agent Platform Pipelines with KFP: Components, Templates & Caching 15.3 Ensuring Consistent Preprocessing Between Training & Serving
16 Chapter 16: Orchestration with Managed Airflow & Ray
16.1 Orchestrating ML Workflows with Managed Service for Apache Airflow 16.2 Ray on Agent Platform for Distributed ML Workloads
17 Chapter 17: Automating Retraining & CI/CD/CT
17.1 Choosing a Retraining Policy 17.2 CI/CD/CT for ML with Cloud Build
18 Chapter 18: Securing AI Systems, Responsible AI & Explainability
18.1 Protecting AI Systems from Data & Model Exfiltration 18.2 Defending Gen AI Apps: Safety Filters, Model Armor & Regex Guardrails 18.3 Responsible AI: Detecting & Monitoring Bias 18.4 Model Explainability on Agent Platform
19 Chapter 19: Monitoring AI Solutions
19.1 Configuring Model Monitoring on Agent Platform 19.2 Training-Serving Skew, Data Drift, Concept Drift & Attribution Drift 19.3 Monitoring, Testing & Evaluating Gen AI Solutions in Production
17.1 Choosing a Retraining Policy
Key Takeaways Explain This Topic AI
Google's MLOps guidance lists retraining triggers on demand, on a schedule, on availability of new training data, on model performance degradation, and on significant changes in data distributions.
Schedule-based retraining fits when new labeled data arrives predictably, while degradation-based retraining needs ground truth or reliable drift signals.
Continuous training (CT) automatically retrains and validates models in production, which Google describes as MLOps level 1.
Label delay limits performance-triggered retraining: fraud or churn outcomes can take weeks, so drift on input features often serves as an earlier proxy signal.
Every retraining policy still needs validation gates, because automatic retraining on bad data can deploy a worse model.
Last updated: September 2026
The exam guide lists determining an appropriate retraining policy . The policy answers three questions: when to retrain, on what data , and what must pass before the new model replaces the old one.
Why Models Need Retraining
Change Example Effect
Data drift (input distribution shifts) A new customer segment from a marketing campaign The model sees unfamiliar inputs
Concept drift (the relationship between inputs and label changes) Fraudsters change tactics, so the same features now mean something different Accuracy drops even if inputs look similar
- Seasonality and events Holiday demand, pandemics, price changes Past patterns stop holding
- Upstream changes A new data source or changed feature definition Silent skew or breakage
- Business changes New products, policies, or markets Model outputs no longer match needs
- Retraining Triggers
Google's MLOps guidance lists these pipeline triggers:
Trigger How it works on Google Cloud Best when Watch out for
On demand Manually run the pipeline or template Early stages, investigations, one-off fixes Depends on people remembering
On a schedule Scheduler API cron (daily, weekly, monthly) or a Managed Airflow DAG New labeled data arrives on a predictable cadence, such as weekly sales Wastes compute when nothing changed, and misses sudden shifts between runs
On availability of new training data Eventarc trigger on BigQuery inserts, Pub/Sub message from ingestion, or an Airflow sensor Data arrives irregularly, such as a monthly label file from a partner Tiny data increments may not justify retraining
On performance degradation Continuous evaluation with ground truth detects metric drops → alert → Pub/Sub → pipeline Labels arrive quickly enough to measure accuracy Needs labels, and label delay slows detection
On significant distribution change Model Monitoring drift or skew alert → Pub/Sub → pipeline Labels are slow, and input drift is a useful early signal Not all drift hurts accuracy, so combine with evaluation gates
- Factors That Shape the Policy
- Label availability and delay
Fast labels (clicks, next-day demand): monitor accuracy directly and retrain on degradation or a short schedule.
Slow labels (loan defaults after months, churn after a billing cycle): use input drift and prediction drift as early warnings, and retrain on a schedule timed to label arrival.
Rate of change
- Domain Typical cadence
- Ad click-through, news recommendations Daily or continuous
- Retail demand Weekly, plus event-driven retraining before major promotions
- Credit risk Quarterly or semiannual, with strict validation and governance
- Image defect detection on a stable line Retrain when the line, product, or camera changes
- Cost and risk
Each retraining run costs compute (training, evaluation, tuning) and review time.
Each deployment carries regression risk , which is why validation gates and canaries are part of the policy (Chapters 13 and 15).
Regulated models often need human approval after automated validation.
What Data to Retrain On
- Strategy Description Use when
- Sliding window Most recent N months only Old behavior is no longer relevant
- Expanding window All history to date Patterns are stable and rare events matter
- Weighted recency All data, with recent rows weighted more Balance between stability and adaptation
Incremental / warm start Continue training from the previous model on new data Large models where full retraining is expensive. Watch for forgetting older patterns
Continuous tuning (Gemini) Tune further from a tuned model or checkpoint with new examples Gen AI tasks with new labeled examples (Chapter 10)
Keep a fixed, time-appropriate evaluation set , plus the most recent period, so successive models are compared fairly.
Full Retrain vs. Re-tune vs. Redesign
- Situation Action
- Same features, fresh data, mild drift Retrain with the existing pipeline and hyperparameters
- Larger shifts or noticeably lower validation performance Retrain plus hyperparameter tuning
New signals needed, or the concept fundamentally changed Redesign features or model. This is a new experiment cycle, not automatic CT
Designing a Retraining Policy: Worked Example
An online retailer's demand forecasting model:
Labels: actual sales arrive daily, so accuracy (WAPE) can be computed weekly.
Baseline trigger: a weekly schedule every Monday after the previous week's sales load, with a data-freshness sensor.
Event trigger: WAPE on the last 7 days more than 15% worse than the validation benchmark → alert → Pub/Sub → an ad hoc pipeline run .
Early warning: Model Monitoring drift alerts on price and promotion features notify the team but don't retrain automatically.
Data: 2-year sliding window, with holiday periods kept from prior years.
Gates: schema and statistics validation, champion comparison on the last 8 weeks, per-category checks, then a canary rollout.
Continuous Training in the MLOps Maturity Model
Level Characteristics
Level 0: Manual Data scientists train and hand off models manually. Releases are infrequent. There's no CT and no production monitoring feedback
Level 1: ML pipeline automation An automated training pipeline is deployed to production. Continuous training runs on triggers, with data and model validation, feature store (optional), and metadata management
Level 2: CI/CD pipeline automation Adds automated build, test, and deployment of the pipeline itself , so new ML ideas (features, architectures) reach production quickly (Section 17.2)