Study guide
2.10 Experiment Tracking, Artifact Lineage and Model Registry Governance
Google Cloud Professional Machine Learning Engineer Exam
Track your progress as you read
Finished sections get a check mark.
1 Introduction & Exam Overview
0.1 Exam Overview, Blueprint & Certification Logistics 0.2 Study Strategy, Scenario Mechanics & 8-Week Roadmap
2 Section 1: Architecting Low-Code AI Solutions (~13%)
1.1 BigQuery ML: Model Types, SQL Syntax and In-Database Evaluation 1.2 Feature Engineering and Feature Selection in BigQuery ML 1.3 Generating Predictions with BigQuery ML 1.4 AutoML on Agent Platform: Tabular, Image, Text and Video Training 1.5 Fine-Tuning Gemini Models from BigQuery 1.6 Model Garden: Evaluating and Selecting the Right Model 1.7 Industry-Specific AI APIs: Document AI, Vision, Translate and Speech 1.8 Generative Media and Models as a Service: Gemini, Imagen and Veo 1.9 Optimizing Gemini Applications for Cost, Latency and Availability
3 Section 2: Collaborating Within and Across Teams to Manage Data and Models (~16%)
2.1 Organizing and Exploring Tabular, Text, Image and Video Data 2.2 Choosing the Preprocessing Tool: BigQuery, Dataflow, Spark and In-Memory Python 2.3 Creating and Consolidating Features in Agent Platform Feature Store 2.4 Data Privacy, PII Handling and Sensitive Information 2.5 Notebook Environments: Collaboration and Security Best Practices 2.6 Prototyping in Workbench and Colab Enterprise with PyTorch, scikit-learn and JAX 2.7 Prototyping with Model Garden Foundational and Open Models in Notebooks 2.8 Choosing the Development and Experimentation Environment 2.9 Evaluating Generative Solutions: LLM-as-a-Judge and RAG Evaluation 2.10 Experiment Tracking, Artifact Lineage and Model Registry Governance
4 Section 3: Scaling Prototypes into ML Models (~21%)
3.1 Choosing the Model Type: ARIMA, DNN, LLM and Classical Baselines 3.2 Choosing the Product: AutoML, BigQuery ML, Custom Training and Pipelines 3.3 Choosing the Deployment Strategy at Design Time 3.4 Modeling Techniques for Interpretability Requirements 3.5 Organizing Training Data on Cloud Storage and BigQuery 3.6 Ingesting Structured and Unstructured Data into Training Pipelines 3.7 Training SDKs and Services: Custom Training, Kubeflow on GKE, AutoML and Tabular Workflows 3.8 Troubleshooting ML Model Training Failures 3.9 Hyperparameter Tuning, Bayesian Optimization and Vizier 3.10 Model Evaluation Metrics, Transfer Learning and Fine-Tuning 3.11 Evaluating Compute and Accelerator Options: CPU, GPU and TPU 3.12 Distributed Training Strategies, Hardware Acceleration and Performance Optimization
5 Section 4: Serving and Scaling Models (~20%)
4.1 Batch and Online Inference: Agent Platform, Model Garden, Cloud Run and GKE 4.2 Rollout Strategies: Canary, Traffic Splitting, Shadow and Blue-Green 4.3 Tuning Models for Production: Vector Search and Model Compression 4.4 Deploying Models to Public and Private Endpoints 4.5 Choosing Serving Hardware: CPU, GPU, TPU and Edge 4.6 Scaling the Serving Backend for Throughput
6 Section 5: Automating and Orchestrating ML Pipelines (~18%)
5.1 Validating Data and Models Inside Pipelines 5.2 Agent Platform Pipelines: Kubeflow Pipelines (KFP) and TFX Architecture 5.3 Reusable Pipeline Components, Artifacts and ML Metadata Lineage 5.4 Ray on Agent Platform for Distributed Python Workloads 5.5 MLOps CI/CD Automation, Retraining Triggers and Skew Prevention
7 Section 6: Monitoring AI Solutions (~13%)
6.1 Securing AI Systems: Exfiltration, Malicious Prompting and Model Armor 6.2 Responsible AI, Explainable AI, Fairness Metrics and Model Governance 6.3 Model Monitoring: Feature Skew and Drift Detection 6.4 Operational Troubleshooting, Performance Profiling and Cost Optimization 6.5 Monitoring, Testing and Evaluating Generative AI in Production
2.10 Experiment Tracking, Artifact Lineage and Model Registry Governance
Key Takeaways Explain This Topic AI
Experiments on Agent Platform records parameters, metrics, and artifacts per run so attempts are comparable rather than lost in notebook scrollback.
ML Metadata stores artifacts, executions, contexts, and the edges between them, which is what makes "which data produced this model" answerable during an audit.
The Model Registry holds versioned models with mutable aliases such as default and champion, so a rollback is an alias change rather than a redeployment.
An experiment run is not a model version — promoting a run means explicitly registering its artifact as a version in the registry.
Model cards attached to registry entries record intended use, training data, evaluation results, and limitations for governance review.
Last updated: September 2026
2.10 Experiment Tracking, Artifact Lineage and Model Registry Governance
Blueprint reference: Section 2.3, "Tracking and comparing model artifacts, versions, and lineage (e.g., Experiments on Agent Platform and Gemini Enterprise Agent Platform ML Metadata)."
Moving machine learning from experimental exploration to regulated, repeatable production requires systematic experiment tracking, artifact lineage, and rigorous model lifecycle governance. On Google Cloud, Experiments on Agent Platform , Agent Platform ML Metadata , and the Model Registry carry those three responsibilities.
1. Where This Section Sits
Notebook surfaces themselves — Agent Platform Workbench versus Colab Enterprise, runtime templates, least-privilege service accounts, private IP with IAP, and idle shutdown — are covered in Sections 2.5 and 2.6 . This section picks up after the prototype works and asks the governance question: how is the run recorded, how is the resulting artifact catalogued, and how is its provenance proved later.
Three services answer that, and the exam expects you to keep their roles distinct:
- Service Answers Scope
- Experiments on Agent Platform "What did we try, with what parameters, and how did it score?" Runs and metrics
Agent Platform ML Metadata "What produced this artifact, and what did it produce in turn?" Artifacts, executions, lineage
Model Registry "Which model versions exist, which is live, and how do we roll back?" Model versions and aliases
A common wrong answer conflates the first and third: Experiments compares attempts , the Model Registry catalogues shippable versions . An experiment run is not a model version, and promoting one means explicitly registering it.
2. Systematic Tracking with Vertex AI Experiments & TensorBoard
In iterative ML development, data scientists experiment with multiple architectures, optimizers, learning rates, and feature sets. Ad-hoc spreadsheet logging fails to provide reproducibility or lineage. +---------------------------------------------------------------------------------------+
- | VERTEX AI EXPERIMENTS DATA MODEL |
- | |
- | Experiment: `fraud-detection-transformer` |
- | ├── Experiment Run: `run-lr-001-batch-64` |
- | │ ├── Parameters: {learning_rate: 0.001, batch_size: 64, optimizer: 'Adam'} |
- | │ ├── Metrics: {val_loss: 0.214, pr_auc: 0.942, f1_score: 0.891} |
- | │ ├── Artifacts: `gs://my-bucket/artifacts/model_v1.tar.gz` |
- | │ └── TensorBoard Time Series: Managed TensorBoard Instance Log |
- | └── Experiment Run: `run-lr-0001-batch-128` |
- | ├── Parameters: {learning_rate: 0.0001, batch_size: 128, optimizer: 'AdamW'} |
- | └── Metrics: {val_loss: 0.187, pr_auc: 0.965, f1_score: 0.923} |
- +---------------------------------------------------------------------------------------+
- Key Capabilities of Vertex AI Experiments
Parameters & Metrics Tracking : Using the google-cloud-aiplatform Python SDK, data scientists log hyperparameters ( aiplatform.log_params() ) and scalar evaluation metrics ( aiplatform.log_metrics() ) directly from code.
Artifact & Lineage Recording : Connects input datasets (BigQuery tables, GCS CSVs) to intermediate model checkpoints and output evaluation confusion matrices in Vertex ML Metadata .
Managed Vertex AI TensorBoard : Google Cloud provides a fully managed, enterprise-scale TensorBoard service without requiring local daemon hosting. Features include:
Real-time loss and accuracy curve visualization across multiple runs.
Histogram of weights and gradients to diagnose vanishing/exploding gradients.
Profiling GPU memory utilization and compute bottlenecks during training execution.
3. Enterprise Model Governance: Vertex AI Model Registry
Vertex AI Model Registry serves as the central catalog for trained machine learning models across an organization. It bridges the gap between model development and production serving. VERTEX AI MODEL REGISTRY ECOSYSTEM
- |
- +--------------------------------------+--------------------------------------+
- | |
- [ Model Lifecycle & Versioning ] [ Governance & Deployment ]
- | |
- Immutable Model Versions (v1, v2, v3) - Vertex Model Cards (Ethics & Bias)
- Mutable Version Aliases (@default, @prod) - Deployment Lineage (Vertex Endpoints)
- Custom Container Image Specifications - Cross-Project Sharing (Shared VPC / IAM)
- Pre-built Serving Containers (TF, PyTorch, SKLearn) - Batch Prediction & Endpoint Deployments
- Model Versioning and Mutable Aliases
Immutable Versioning : Every time a model is registered or updated with a new artifact, Vertex AI Model Registry assigns an incremental, immutable version number ( 1 , 2 , 3 ). The underlying artifact URI, container image, and schema cannot be altered.
Mutable Version Aliases : Aliases act as symbolic tags pointing to specific model versions. Common alias patterns include @default , @staging , @canary , and @production .
When a model passes automated evaluation gates, engineers reassign the @production alias from Version 1 to Version 2.
Upstream production inference clients querying @production automatically receive predictions from the new model version without modifying client code or endpoint URL configurations.
Model Cards for Governance and Responsible AI
Vertex AI Model Registry integrates Vertex Model Cards , structured documentation artifacts that record critical operational and ethical characteristics of the model:
Intended Use Cases : Approved production applications and explicitly out-of-scope or prohibited use cases.
Training & Evaluation Data Lineage : Datasets utilized, demographic splits, and historical data timeframes.
Performance & Fairness Benchmarks : Accuracy, precision-recall metrics broken down across protected demographic slices, and fairness metrics (e.g., disparate impact ratio).
Maintenance Schedule : Model expiration dates and mandatory retraining schedules.
Cross-Project Model Sharing & IAM Governance
In multi-project enterprise architectures (e.g., ml-dev-project , ml-staging-project , ml-prod-project ):
Models registered in a centralized governance project can be shared with downstream production deployment projects using granular IAM permissions ( roles/aiplatform.viewer and roles/aiplatform.user ).
Model artifacts stored in Cloud Storage buckets can be locked down using Customer-Managed Encryption Keys (CMEK) and accessed across projects via service accounts.