2.7 Prototyping with Model Garden Foundational and Open Models in Notebooks

Google Cloud Professional Machine Learning Engineer Exam

Track your progress as you read

Finished sections get a check mark.

1 Introduction & Exam Overview

0.1 Exam Overview, Blueprint & Certification Logistics 0.2 Study Strategy, Scenario Mechanics & 8-Week Roadmap

2 Section 1: Architecting Low-Code AI Solutions (~13%)

1.1 BigQuery ML: Model Types, SQL Syntax and In-Database Evaluation 1.2 Feature Engineering and Feature Selection in BigQuery ML 1.3 Generating Predictions with BigQuery ML 1.4 AutoML on Agent Platform: Tabular, Image, Text and Video Training 1.5 Fine-Tuning Gemini Models from BigQuery 1.6 Model Garden: Evaluating and Selecting the Right Model 1.7 Industry-Specific AI APIs: Document AI, Vision, Translate and Speech 1.8 Generative Media and Models as a Service: Gemini, Imagen and Veo 1.9 Optimizing Gemini Applications for Cost, Latency and Availability

3 Section 2: Collaborating Within and Across Teams to Manage Data and Models (~16%)

2.1 Organizing and Exploring Tabular, Text, Image and Video Data 2.2 Choosing the Preprocessing Tool: BigQuery, Dataflow, Spark and In-Memory Python 2.3 Creating and Consolidating Features in Agent Platform Feature Store 2.4 Data Privacy, PII Handling and Sensitive Information 2.5 Notebook Environments: Collaboration and Security Best Practices 2.6 Prototyping in Workbench and Colab Enterprise with PyTorch, scikit-learn and JAX 2.7 Prototyping with Model Garden Foundational and Open Models in Notebooks 2.8 Choosing the Development and Experimentation Environment 2.9 Evaluating Generative Solutions: LLM-as-a-Judge and RAG Evaluation 2.10 Experiment Tracking, Artifact Lineage and Model Registry Governance

4 Section 3: Scaling Prototypes into ML Models (~21%)

3.1 Choosing the Model Type: ARIMA, DNN, LLM and Classical Baselines 3.2 Choosing the Product: AutoML, BigQuery ML, Custom Training and Pipelines 3.3 Choosing the Deployment Strategy at Design Time 3.4 Modeling Techniques for Interpretability Requirements 3.5 Organizing Training Data on Cloud Storage and BigQuery 3.6 Ingesting Structured and Unstructured Data into Training Pipelines 3.7 Training SDKs and Services: Custom Training, Kubeflow on GKE, AutoML and Tabular Workflows 3.8 Troubleshooting ML Model Training Failures 3.9 Hyperparameter Tuning, Bayesian Optimization and Vizier 3.10 Model Evaluation Metrics, Transfer Learning and Fine-Tuning 3.11 Evaluating Compute and Accelerator Options: CPU, GPU and TPU 3.12 Distributed Training Strategies, Hardware Acceleration and Performance Optimization

5 Section 4: Serving and Scaling Models (~20%)

4.1 Batch and Online Inference: Agent Platform, Model Garden, Cloud Run and GKE 4.2 Rollout Strategies: Canary, Traffic Splitting, Shadow and Blue-Green 4.3 Tuning Models for Production: Vector Search and Model Compression 4.4 Deploying Models to Public and Private Endpoints 4.5 Choosing Serving Hardware: CPU, GPU, TPU and Edge 4.6 Scaling the Serving Backend for Throughput

6 Section 5: Automating and Orchestrating ML Pipelines (~18%)

5.1 Validating Data and Models Inside Pipelines 5.2 Agent Platform Pipelines: Kubeflow Pipelines (KFP) and TFX Architecture 5.3 Reusable Pipeline Components, Artifacts and ML Metadata Lineage 5.4 Ray on Agent Platform for Distributed Python Workloads 5.5 MLOps CI/CD Automation, Retraining Triggers and Skew Prevention

7 Section 6: Monitoring AI Solutions (~13%)

6.1 Securing AI Systems: Exfiltration, Malicious Prompting and Model Armor 6.2 Responsible AI, Explainable AI, Fairness Metrics and Model Governance 6.3 Model Monitoring: Feature Skew and Drift Detection 6.4 Operational Troubleshooting, Performance Profiling and Cost Optimization 6.5 Monitoring, Testing and Evaluating Generative AI in Production

2.7 Prototyping with Model Garden Foundational and Open Models in Notebooks

Key Takeaways Explain This Topic AI

Model Garden entries ship with notebook artifacts that either call a managed API or deploy an open checkpoint to a temporary endpoint for evaluation.

Prototyping open-weight models requires an endpoint with accelerators, which bills continuously — undeploy it as soon as the comparison is finished.

A like-for-like comparison needs the same eval set, the same prompts, and recorded latency and cost alongside the quality score, all logged to Experiments.

Parameter-efficient adapter tuning lets a notebook prototype a customized open model without full-parameter retraining.

A prototype only becomes a decision when the comparison is reproducible: eval data versioned, prompts stored, and results attached to an experiment run.

Last updated: September 2026

2.7 Prototyping with Model Garden Foundational and Open Models in Notebooks

Blueprint reference: Section 2.2, "Using a variety of foundational and open-source models in Model Garden to create model prototypes in notebook environments."

The previous chapter covered selecting a model. This bullet is about the mechanics of the comparison itself, and the exam's angle is almost always cost or reproducibility rather than modelling.

Two Very Different Prototyping Paths

Path A — managed API models. Gemini, other first-party models, and partner models as a service are called through the SDK. There is nothing to deploy, billing is per token, and a notebook can compare three of them in minutes for a few cents. from vertexai.generative_models import GenerativeModel # SDK package name is unchanged

model = GenerativeModel("gemini-2.5-flash")
response = model.generate_content(prompt)

Path B — open-weight checkpoints. Gemma, Llama-family, and other open models in Model Garden are deployed : the notebook provisions an endpoint with an accelerator, uploads or references the checkpoint, and waits for the deployment to become ready. Only then can it send requests.

The exam's favourite consequence: that endpoint bills for accelerator hours from the moment it is ready until it is undeployed , regardless of whether the notebook sends a single request. A prototype endpoint left up over a weekend is a well-known and entirely avoidable cost incident. The correct habit is to undeploy the model and delete the endpoint at the end of the comparison, and to build that teardown into the notebook itself rather than trusting memory. # Always paired with the deploy call, ideally in a try/finally

endpoint.undeploy_all()
endpoint.delete()

A second consequence is quota . Deploying a large open model requires accelerator quota in the region, and "the deployment never becomes ready" is usually a quota problem rather than a model problem.

Structuring a Fair Comparison

A prototype that cannot be defended is not a decision. The elements of a defensible comparison:

A fixed eval set. The same 100–300 representative inputs for every candidate, versioned so the comparison can be repeated. Include the hard cases deliberately; an eval set of easy examples separates nothing.

Identical prompts. If one candidate gets a carefully engineered prompt and another gets a first draft, the comparison measures prompt effort, not model quality.

Quality scoring. Task metrics where ground truth exists; an LLM-as-a-judge rubric where it does not, with the rubric written down.

Latency and cost recorded on the same runs. p50 and p95 latency, tokens consumed, and — for self-deployed models — the accelerator configuration and hourly rate.

Everything logged to Experiments. Each candidate is a run with parameters (model name, temperature, prompt version) and metrics (score, latency, cost). This turns scrollback into a comparable record. import vertexai

vertexai.init(experiment="model-selection-support-router")

with vertexai.start_run(run="gemma-open-endpoint-a2"):

vertexai.log_params({"model": "gemma", "accelerator": "1xL4", "prompt_v": 3})
vertexai.log_metrics({"judge_score": 0.86, "p95_ms": 410})
Lightweight Customization in a Notebook

Full-parameter fine-tuning of a large open model is a training-job workload, not a notebook workload. What is practical in a notebook is parameter-efficient tuning — training a small set of adapter weights on top of a frozen base model. The base checkpoint stays unchanged, the trained artifact is small, and multiple adapters can be maintained against one base model.

This gives a prototyping loop that is genuinely fast: try prompting first, then few-shot, then a small adapter, and only escalate to full tuning or a larger model if the adapter fails to close the gap. It is the same escalation ladder as the Gemini tuning decision, applied to open models.

Prototype to Production

The handoff points the exam cares about:

Register the winner. Upload the chosen model to the Model Registry with a version and a description of why it won, so serving and governance work from a catalogued artifact rather than a notebook variable.

Keep the eval set. It becomes the regression suite for future model upgrades. When a base model version changes, rerunning the same eval set is the only way to know whether quality moved.

Move the loop into a component. A comparison worth repeating belongs in a pipeline so that the next model release is evaluated automatically rather than by whoever remembers.

Record the negative results. Which models were rejected and why is the part of the record that prevents the same comparison being run again in six months.

Exam Traps

Leaving a prototype endpoint deployed. The single most common cost mistake in this area.

Comparing models on different prompts or different eval data. The comparison measures nothing.

Assuming open models are cheaper. Free weights, paid accelerators.

Full fine-tuning inside a notebook kernel. Use a training job.

Discarding the eval set after the decision. It is the regression suite.