Study guide
10.3 Fine-Tuning Foundation Models & When to Tune
Google Cloud Professional Machine Learning Engineer PMLE
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Exam Overview & the Agent Platform Map
1.1 PMLE Exam Facts, Policies & the June 2026 Blueprint 1.2 From Vertex AI to Gemini Enterprise Agent Platform: The Product Name Map
2 Chapter 2: Low-Code ML with BigQuery ML
2.1 Choosing, Training & Predicting with BigQuery ML Models 2.2 Feature Engineering & Feature Selection in BigQuery ML 2.3 Fine-Tuning & Evaluating Gemini Models from BigQuery
3 Chapter 3: AutoML & Google Cloud AI APIs
3.1 Training Models with Agent Platform AutoML 3.2 Building with Document AI, Vision, Translation & Other AI APIs
4 Chapter 4: Foundation Models in Model Garden
4.1 Evaluating & Selecting Models in Model Garden 4.2 Building & Tuning Solutions with Gemini, Imagen, Veo & MaaS 4.3 Optimizing Gemini Applications for Cost, Latency & Availability
5 Chapter 5: Exploring, Preprocessing & Protecting Data
5.1 Organizing & Exploring Tabular, Text & Image Data 5.2 Choosing a Preprocessing Tool: BigQuery, Dataflow, Spark or Python 5.3 Creating & Consolidating Features in Agent Platform Feature Store 5.4 Protecting PII & Sensitive Data in ML Workflows
6 Chapter 6: Notebooks, Prototyping & Experiment Tracking
6.1 Workbench vs. Colab Enterprise: Secure, Collaborative Notebooks 6.2 Prototyping with PyTorch, scikit-learn, JAX & Model Garden in Notebooks 6.3 Experiment Environments, Tracking & Lineage with Experiments and ML Metadata
7 Chapter 7: Evaluating Predictive & Generative AI Solutions
7.1 Evaluation Metrics for Predictive Models 7.2 Evaluating Gen AI: Computation Metrics, Rubrics & LLM-as-a-Judge
8 Chapter 8: Choosing the Modeling Approach
8.1 Choosing the Model Type: ARIMA, Trees, DNNs & LLMs 8.2 Choosing the Product & Deployment Strategy 8.3 Modeling Techniques for Interpretability Requirements
9 Chapter 9: Training Data & Training SDKs
9.1 Organizing & Ingesting Training Data from Cloud Storage & BigQuery 9.2 Custom Training on Agent Platform: Containers, Jobs & Worker Pools 9.3 Kubeflow on GKE, AutoML & Tabular Workflows as Training Choices
10 Chapter 10: Troubleshooting, Hyperparameter Tuning & Fine-Tuning
10.1 Troubleshooting ML Training Failures 10.2 Hyperparameter Tuning on Agent Platform 10.3 Fine-Tuning Foundation Models & When to Tune
11 Chapter 11: Training Hardware & Distributed Training
11.1 Choosing CPUs, GPUs & TPUs for Training 11.2 Distributed Training: Data & Model Parallelism
12 Chapter 12: Deploying Models for Inference
12.1 Batch vs. Online Inference on Agent Platform 12.2 Serving Beyond Agent Platform Endpoints: Model Garden, Cloud Run & GKE 12.3 Prebuilt & Custom Serving Containers
13 Chapter 13: Model Versions, Rollouts & Inference Logic
13.1 Organizing & Versioning Models in Model Registry 13.2 Rollout Strategies: A/B Tests, Canaries & Traffic Splits 13.3 Inference Preprocessing & Postprocessing
14 Chapter 14: Scaling Online Serving
14.1 Serving Features Online with Feature Store 14.2 Public, Dedicated & Private Endpoints 14.3 Serving Hardware & Autoscaling for Throughput 14.4 Tuning Models for Production Training & Serving
15 Chapter 15: End-to-End ML Pipelines
15.1 Validating Data & Models in Pipelines 15.2 Agent Platform Pipelines with KFP: Components, Templates & Caching 15.3 Ensuring Consistent Preprocessing Between Training & Serving
16 Chapter 16: Orchestration with Managed Airflow & Ray
16.1 Orchestrating ML Workflows with Managed Service for Apache Airflow 16.2 Ray on Agent Platform for Distributed ML Workloads
17 Chapter 17: Automating Retraining & CI/CD/CT
17.1 Choosing a Retraining Policy 17.2 CI/CD/CT for ML with Cloud Build
18 Chapter 18: Securing AI Systems, Responsible AI & Explainability
18.1 Protecting AI Systems from Data & Model Exfiltration 18.2 Defending Gen AI Apps: Safety Filters, Model Armor & Regex Guardrails 18.3 Responsible AI: Detecting & Monitoring Bias 18.4 Model Explainability on Agent Platform
19 Chapter 19: Monitoring AI Solutions
19.1 Configuring Model Monitoring on Agent Platform 19.2 Training-Serving Skew, Data Drift, Concept Drift & Attribution Drift 19.3 Monitoring, Testing & Evaluating Gen AI Solutions in Production
10.3 Fine-Tuning Foundation Models & When to Tune
Key Takeaways Explain This Topic AI
Google recommends finding the best prompt first and tuning only when prompting can't reach the quality, consistency, or cost target.
Gemini supervised tuning jobs report training and validation loss and fraction_of_correct_next_step_preds so you can spot overfitting.
Tuning jobs with fewer than 10 epochs save about one checkpoint per epoch, and longer jobs save about 10 evenly spaced checkpoints plus the final one.
Continuous tuning keeps tuning an already tuned Gemini model or checkpoint with more epochs or new examples.
For open models, QLoRA uses about 75% less peak GPU memory than LoRA, while LoRA tunes about 66% faster and costs up to 40% less.
Last updated: September 2026
The exam guide lists fine-tuning foundational models from Agent Platform and Model Garden, and when tuning should be considered . Chapter 4 compared tuning methods (supervised, preference, reinforcement, distillation). This section covers the decision and the execution.
When Should Tuning Be Considered?
Google's guidance: start with prompting to find the best prompt, then tune if needed to boost performance or fix recurring errors.
- Signal Tune? Why
- Well-written prompts still give inconsistent format, tone, or labels Yes Tuning teaches the behavior directly
Long few-shot prompts make every request slow and expensive Yes A tuned model needs shorter prompts, cutting tokens and latency
A task with domain-specific syntax or rules (claim codes, internal query language) Yes The examples encode patterns that are hard to explain in a prompt
Answers need current or changing facts No, ground instead Tuned knowledge goes stale, while retrieval stays current
Only a handful of examples exist Not yet Supervised tuning works best with a sizable, high-quality set (think 100+ examples, usually hundreds)
- Requirements change weekly Probably not Each change means rebuilding data and re-tuning
- The base model already meets the quality bar No Tuning adds cost and lifecycle work
Before tuning, analyze failures . If errors come from missing context, fix retrieval. If they come from ambiguous instructions, fix the prompt. Tune when the model needs to learn a pattern .
- Running a Gemini Supervised Tuning Job
- 1. Prepare data
Training dataset: JSON Lines in Cloud Storage. Each example holds the conversation ( user input and ideal model output), optionally with a system instruction. Multimodal examples reference files by URI.
Validation dataset (optional but recommended): separate examples from the same distribution. This enables validation metrics.
Match production prompt formats exactly, and remove PII and duplicates.
2. Configure the job
- Setting Effect
- Base model A Gemini version that supports supervised tuning
- Epochs Full passes over the training data. Leave unset to use the recommended value
Adapter size Number of trainable parameters in the adapter. Larger sizes can learn more complex tasks but need more data and compute
- Learning rate multiplier Scales the recommended learning rate
- Export last checkpoint only Skip intermediate checkpoints
- Region Tuning and tuned-model serving run only in the regions each model supports
- 3. Monitor tuning metrics
- Metric Meaning
- /train_total_loss , /eval_total_loss Loss on the tuning and validation data
/train_fraction_of_correct_next_step_preds , /eval_fraction_of_correct_next_step_preds Token-level accuracy against the ground truth
/train_num_predictions , /eval_num_predictions Tokens predicted per step
If training loss keeps falling while validation loss rises , the model is overfitting.
4. Use checkpoints
Fewer than 10 epochs: about one checkpoint per epoch .
More than 10 epochs: about 10 evenly spaced checkpoints , plus the final checkpoint.
Intermediate checkpoints are deployed to endpoints as tuning progresses. You can pick the best checkpoint before overfitting as the default.
5. Deploy and evaluate
The tuning job produces a tuned model with an endpoint for inference. Tuned-model inference shares quota with the base model . Evaluate the tuned model against the base model on the same evaluation set with Gen AI evals (Chapter 7) before switching traffic. For thinking-capable models, set the thinking level to MINIMAL (Gemini 3 and later) or a thinking budget of 0 (Gemini 2.5 and earlier) on tuned tasks.
6. Continue tuning when needed
Continuous tuning starts from an already tuned model or checkpoint and adds epochs or new examples. It helps when a model underfits, when new data arrives, or when you want further customization, without starting over from the base model.
Tuning Open Models from Model Garden
Open models (such as Gemma or Llama) can be tuned in several ways:
The model card's fine-tuning pipeline (runs on Agent Platform Pipelines) or Open notebook option.
Custom training jobs with your own code.
Supervised modes: full fine-tuning (all weights, highest potential quality and cost) or LoRA (parameter-efficient adapters). Distillation tunes a smaller student on a larger teacher's outputs.
LoRA vs. QLoRA
- Consideration Recommendation
- Least GPU memory QLoRA : about 75% smaller peak GPU memory
- Fastest tuning LoRA : about 66% faster
- Lowest cost LoRA : up to 40% less expensive
- Longer max sequence length or bigger batches QLoRA (less memory per example)
- Accuracy improvement About the same for both
Example from Google's guidance: tuning openLLaMA-7B on one L4 GPU fails with out-of-memory at LoRA batch size 1, while QLoRA runs at batch size 12. If you only have smaller GPUs, QLoRA makes tuning possible.
Tuned open-model weights are self-deployed from Model Garden to dedicated endpoints (billed for compute), with prebuilt serving containers such as vLLM that can serve adapters.
Cost and Latency Effects of Tuning
Tuning cost is charged per tuning run, based on the tokens processed during tuning. Budget for several iterations, not one.
Inference savings come from shorter prompts . Removing a 3,000-token few-shot block from millions of requests often outweighs the tuning cost.
Smaller tuned models (Flash-Lite, or distilled open models) can match a larger prompted model on a narrow task at lower latency.
Self-deployed open models trade per-token pricing for endpoint compute cost, which pays off at steady, high volume.
Tuning Lifecycle Checklist
Base model version and its retirement date recorded.
Training and validation datasets versioned in Cloud Storage.
Tuning job configuration logged as an experiment run.
Best checkpoint chosen from validation metrics.
Evaluation results against the base model stored with the model in Model Registry.
Re-tuning planned before the base model retires.