Study guide
2.3 Fine-Tuning & Evaluating Gemini Models from BigQuery
Google Cloud Professional Machine Learning Engineer PMLE
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Exam Overview & the Agent Platform Map
1.1 PMLE Exam Facts, Policies & the June 2026 Blueprint 1.2 From Vertex AI to Gemini Enterprise Agent Platform: The Product Name Map
2 Chapter 2: Low-Code ML with BigQuery ML
2.1 Choosing, Training & Predicting with BigQuery ML Models 2.2 Feature Engineering & Feature Selection in BigQuery ML 2.3 Fine-Tuning & Evaluating Gemini Models from BigQuery
3 Chapter 3: AutoML & Google Cloud AI APIs
3.1 Training Models with Agent Platform AutoML 3.2 Building with Document AI, Vision, Translation & Other AI APIs
4 Chapter 4: Foundation Models in Model Garden
4.1 Evaluating & Selecting Models in Model Garden 4.2 Building & Tuning Solutions with Gemini, Imagen, Veo & MaaS 4.3 Optimizing Gemini Applications for Cost, Latency & Availability
5 Chapter 5: Exploring, Preprocessing & Protecting Data
5.1 Organizing & Exploring Tabular, Text & Image Data 5.2 Choosing a Preprocessing Tool: BigQuery, Dataflow, Spark or Python 5.3 Creating & Consolidating Features in Agent Platform Feature Store 5.4 Protecting PII & Sensitive Data in ML Workflows
6 Chapter 6: Notebooks, Prototyping & Experiment Tracking
6.1 Workbench vs. Colab Enterprise: Secure, Collaborative Notebooks 6.2 Prototyping with PyTorch, scikit-learn, JAX & Model Garden in Notebooks 6.3 Experiment Environments, Tracking & Lineage with Experiments and ML Metadata
7 Chapter 7: Evaluating Predictive & Generative AI Solutions
7.1 Evaluation Metrics for Predictive Models 7.2 Evaluating Gen AI: Computation Metrics, Rubrics & LLM-as-a-Judge
8 Chapter 8: Choosing the Modeling Approach
8.1 Choosing the Model Type: ARIMA, Trees, DNNs & LLMs 8.2 Choosing the Product & Deployment Strategy 8.3 Modeling Techniques for Interpretability Requirements
9 Chapter 9: Training Data & Training SDKs
9.1 Organizing & Ingesting Training Data from Cloud Storage & BigQuery 9.2 Custom Training on Agent Platform: Containers, Jobs & Worker Pools 9.3 Kubeflow on GKE, AutoML & Tabular Workflows as Training Choices
10 Chapter 10: Troubleshooting, Hyperparameter Tuning & Fine-Tuning
10.1 Troubleshooting ML Training Failures 10.2 Hyperparameter Tuning on Agent Platform 10.3 Fine-Tuning Foundation Models & When to Tune
11 Chapter 11: Training Hardware & Distributed Training
11.1 Choosing CPUs, GPUs & TPUs for Training 11.2 Distributed Training: Data & Model Parallelism
12 Chapter 12: Deploying Models for Inference
12.1 Batch vs. Online Inference on Agent Platform 12.2 Serving Beyond Agent Platform Endpoints: Model Garden, Cloud Run & GKE 12.3 Prebuilt & Custom Serving Containers
13 Chapter 13: Model Versions, Rollouts & Inference Logic
13.1 Organizing & Versioning Models in Model Registry 13.2 Rollout Strategies: A/B Tests, Canaries & Traffic Splits 13.3 Inference Preprocessing & Postprocessing
14 Chapter 14: Scaling Online Serving
14.1 Serving Features Online with Feature Store 14.2 Public, Dedicated & Private Endpoints 14.3 Serving Hardware & Autoscaling for Throughput 14.4 Tuning Models for Production Training & Serving
15 Chapter 15: End-to-End ML Pipelines
15.1 Validating Data & Models in Pipelines 15.2 Agent Platform Pipelines with KFP: Components, Templates & Caching 15.3 Ensuring Consistent Preprocessing Between Training & Serving
16 Chapter 16: Orchestration with Managed Airflow & Ray
16.1 Orchestrating ML Workflows with Managed Service for Apache Airflow 16.2 Ray on Agent Platform for Distributed ML Workloads
17 Chapter 17: Automating Retraining & CI/CD/CT
17.1 Choosing a Retraining Policy 17.2 CI/CD/CT for ML with Cloud Build
18 Chapter 18: Securing AI Systems, Responsible AI & Explainability
18.1 Protecting AI Systems from Data & Model Exfiltration 18.2 Defending Gen AI Apps: Safety Filters, Model Armor & Regex Guardrails 18.3 Responsible AI: Detecting & Monitoring Bias 18.4 Model Explainability on Agent Platform
19 Chapter 19: Monitoring AI Solutions
19.1 Configuring Model Monitoring on Agent Platform 19.2 Training-Serving Skew, Data Drift, Concept Drift & Attribution Drift 19.3 Monitoring, Testing & Evaluating Gen AI Solutions in Production
2.3 Fine-Tuning & Evaluating Gemini Models from BigQuery
Key Takeaways Explain This Topic AI
BigQuery ML tunes Gemini by creating a remote model with an AS SELECT clause that supplies prompt and label columns.
For a tuning remote model, the BigQuery connection's service account needs the Vertex AI Service Agent role in the project that creates the model.
BigQuery ML converts MAX_ITERATIONS to epochs: the number of input rows equals one epoch, and values round up to the nearest whole epoch.
ML.EVALUATE with the TEXT_GENERATION task type returns BLEU4 and ROUGE-L precision, recall, and F1 scores for the model.
Supervised tuning in BigQuery is billed as BigQuery bytes processed plus Agent Platform charges for tuning tokens.
Last updated: September 2026
The June 2026 exam guide lists "fine-tuning Gemini models using BigQuery" under low-code AI. The scenario: an organization's examples (support replies, product descriptions, labeled tickets) already sit in BigQuery, and prompting alone doesn't consistently produce the right style or labels. BigQuery ML runs a supervised tuning job on Agent Platform without the team writing Python or moving data to Cloud Storage.
Step 1: Connect BigQuery to Agent Platform
BigQuery reaches Agent Platform models through a Cloud resource connection . You can reference a specific connection ( project.region.connection_id ) or use DEFAULT . The connection must be in the same location as the dataset that holds the model.
- Remote model use Role for the connection's service account
- Inference with a pre-trained model Agent Platform User
- Supervised tuning Vertex AI Service Agent (in the project where you create the model)
- Step 2: Create a Baseline Remote Model CREATE OR REPLACE MODEL `support.gemini_baseline`
- REMOTE WITH CONNECTION DEFAULT
- OPTIONS (ENDPOINT = 'gemini-2.5-pro');
You can name the model (BigQuery picks the regional endpoint for the dataset's location) or give a full endpoint URL. Supported Gemini models can also use the global endpoint . It improves availability and cuts 429 resource-exhausted errors, but you can't control which region processes the request, so skip it when data-processing location matters.
Run the baseline over held-out rows with AI.GENERATE_TEXT . It is table-valued and supports Gemini, partner, and open models. AI.GENERATE_TABLE returns output that follows a schema you define.
Step 3: Create the Tuned Model
Tuning uses the same CREATE MODEL ... REMOTE statement plus an AS SELECT that supplies training pairs: CREATE OR REPLACE MODEL `support.gemini_tuned`
- REMOTE WITH CONNECTION DEFAULT
- OPTIONS (
- endpoint = 'gemini-2.5-pro',
- prompt_col = 'prompt',
- input_label_cols = ['label'],
- max_iterations = 1500, -- 3 epochs for 500 rows
- learning_rate_multiplier = 1.0,
- data_split_method = 'RANDOM',
- data_split_eval_fraction = 0.1,
- evaluation_task = 'CLASSIFICATION')
- AS
- SELECT CONCAT('Classify this ticket as BILLING, TECH, or ACCOUNT: ', body) AS prompt,
- category AS label
- FROM `support.labeled_tickets`;
Option Meaning
PROMPT_COL / INPUT_LABEL_COLS Columns holding the prompt and the ideal response. The defaults are columns named prompt and label . Both must be STRING
MAX_ITERATIONS Converted to epochs . The number of input rows equals one epoch, so 100 rows and a value of 200 gives two epochs. A value that isn't a multiple rounds up (101 gives two epochs)
LEARNING_RATE_MULTIPLIER Scales the recommended learning rate (default 1.0)
DATA_SPLIT_METHOD AUTO_SPLIT (default), RANDOM , CUSTOM , SEQ , NO_SPLIT . Evaluation rows enable early stopping to reduce overfitting
DATA_SPLIT_EVAL_FRACTION Evaluation fraction for RANDOM or SEQ (default 0.2)
EVALUATION_TASK TEXT_GENERATION , CLASSIFICATION , SUMMARIZATION , QUESTION_ANSWERING , or UNSPECIFIED (default)
With AUTO_SPLIT , fewer than 500 rows all go to training. From 500 to 50,000 rows, 20% is held out for evaluation. Above 50,000 rows, 10,000 rows are held out. After splitting, the training set needs at least 10 rows .
Cost: you pay BigQuery for bytes processed from the training table, and Agent Platform for tokens processed during tuning.
- Step 4: Evaluate Baseline vs. Tuned SELECT * FROM ML.EVALUATE(
- MODEL `support.gemini_tuned`,
- (SELECT prompt AS input_text, label AS output_text FROM `support.holdout_tickets`),
- STRUCT('classification' AS task_type));
- task_type Metrics returned
- TEXT_GENERATION bleu4_score , rouge-l_precision , rouge-l_recall , rouge-l_f1
- CLASSIFICATION precision , recall , f1 per label
- SUMMARIZATION rouge-l_precision , rouge-l_recall , rouge-l_f1
- QUESTION_ANSWERING exact_match (share of outputs identical to the ground truth)
Run the same ML.EVALUATE query against the baseline model and the tuned model on identical holdout rows. Only keep the tuned model if it's clearly better. Not every use case improves with tuning.
Preparing Tuning Data That Actually Helps
Tuning copies patterns from your examples, including mistakes. Before running CREATE MODEL :
Use the production prompt template. If inference will send "Classify this ticket as BILLING, TECH, or ACCOUNT: ...", the training prompts need that same prefix. A template that differs between tuning and inference is a quiet form of training-serving skew.
Make labels exact and consistent. Normalize casing and spelling ( BILLING , not Billing or billing issue ) so the model learns one output form.
Cover the real distribution. Include rare categories, long and short inputs, and edge cases you see in production, not only easy examples.
Remove duplicates and leakage. Duplicate rows inflate effective epochs, and rows copied into both training and holdout inflate evaluation scores.
Hold out a separate evaluation table from the same time period and distribution, so baseline and tuned models are judged on identical inputs.
Redact sensitive data before tuning if the use case doesn't need it (see Chapter 5 on Sensitive Data Protection).
Running the tuned model in production
A tuned remote model is used like any other BigQuery ML model. Scheduled queries or pipeline steps call AI.GENERATE_TEXT or AI.GENERATE_TABLE over new rows and write results back to BigQuery. Because the tuned model is tied to a specific base model version, plan to re-tune and re-evaluate when that base version reaches its retirement date. Treat the evaluation query as a reusable regression test.
Tune, Prompt, or Ground?
Symptom Better first move
Output style, tone, or format is inconsistent despite clear instructions Supervised tuning with examples of the desired output
Classification labels drift or ignore your label set Few-shot prompting, then tuning on labeled examples
Answers need current or proprietary facts that change often Grounding / retrieval-augmented generation (RAG) , not tuning
- Only a handful of examples exist Prompt engineering and few-shot examples, since tuning needs enough good data
- Exam Traps
Recommending an export to Cloud Storage and a Python tuning script when the data is in BigQuery and the team wants low-code SQL.
Forgetting the IAM grant on the connection's service account, a common cause of failed tuning statements.
Judging tuning by a few eyeballed outputs instead of comparing ML.EVALUATE metrics for baseline and tuned models on the same holdout set.
Using the global endpoint when regulations require processing in a specific region.