3.3 Choosing the Deployment Strategy at Design Time

Google Cloud Professional Machine Learning Engineer Exam

Track your progress as you read

Finished sections get a check mark.

1 Introduction & Exam Overview

0.1 Exam Overview, Blueprint & Certification Logistics 0.2 Study Strategy, Scenario Mechanics & 8-Week Roadmap

2 Section 1: Architecting Low-Code AI Solutions (~13%)

1.1 BigQuery ML: Model Types, SQL Syntax and In-Database Evaluation 1.2 Feature Engineering and Feature Selection in BigQuery ML 1.3 Generating Predictions with BigQuery ML 1.4 AutoML on Agent Platform: Tabular, Image, Text and Video Training 1.5 Fine-Tuning Gemini Models from BigQuery 1.6 Model Garden: Evaluating and Selecting the Right Model 1.7 Industry-Specific AI APIs: Document AI, Vision, Translate and Speech 1.8 Generative Media and Models as a Service: Gemini, Imagen and Veo 1.9 Optimizing Gemini Applications for Cost, Latency and Availability

3 Section 2: Collaborating Within and Across Teams to Manage Data and Models (~16%)

2.1 Organizing and Exploring Tabular, Text, Image and Video Data 2.2 Choosing the Preprocessing Tool: BigQuery, Dataflow, Spark and In-Memory Python 2.3 Creating and Consolidating Features in Agent Platform Feature Store 2.4 Data Privacy, PII Handling and Sensitive Information 2.5 Notebook Environments: Collaboration and Security Best Practices 2.6 Prototyping in Workbench and Colab Enterprise with PyTorch, scikit-learn and JAX 2.7 Prototyping with Model Garden Foundational and Open Models in Notebooks 2.8 Choosing the Development and Experimentation Environment 2.9 Evaluating Generative Solutions: LLM-as-a-Judge and RAG Evaluation 2.10 Experiment Tracking, Artifact Lineage and Model Registry Governance

4 Section 3: Scaling Prototypes into ML Models (~21%)

3.1 Choosing the Model Type: ARIMA, DNN, LLM and Classical Baselines 3.2 Choosing the Product: AutoML, BigQuery ML, Custom Training and Pipelines 3.3 Choosing the Deployment Strategy at Design Time 3.4 Modeling Techniques for Interpretability Requirements 3.5 Organizing Training Data on Cloud Storage and BigQuery 3.6 Ingesting Structured and Unstructured Data into Training Pipelines 3.7 Training SDKs and Services: Custom Training, Kubeflow on GKE, AutoML and Tabular Workflows 3.8 Troubleshooting ML Model Training Failures 3.9 Hyperparameter Tuning, Bayesian Optimization and Vizier 3.10 Model Evaluation Metrics, Transfer Learning and Fine-Tuning 3.11 Evaluating Compute and Accelerator Options: CPU, GPU and TPU 3.12 Distributed Training Strategies, Hardware Acceleration and Performance Optimization

5 Section 4: Serving and Scaling Models (~20%)

4.1 Batch and Online Inference: Agent Platform, Model Garden, Cloud Run and GKE 4.2 Rollout Strategies: Canary, Traffic Splitting, Shadow and Blue-Green 4.3 Tuning Models for Production: Vector Search and Model Compression 4.4 Deploying Models to Public and Private Endpoints 4.5 Choosing Serving Hardware: CPU, GPU, TPU and Edge 4.6 Scaling the Serving Backend for Throughput

6 Section 5: Automating and Orchestrating ML Pipelines (~18%)

5.1 Validating Data and Models Inside Pipelines 5.2 Agent Platform Pipelines: Kubeflow Pipelines (KFP) and TFX Architecture 5.3 Reusable Pipeline Components, Artifacts and ML Metadata Lineage 5.4 Ray on Agent Platform for Distributed Python Workloads 5.5 MLOps CI/CD Automation, Retraining Triggers and Skew Prevention

7 Section 6: Monitoring AI Solutions (~13%)

6.1 Securing AI Systems: Exfiltration, Malicious Prompting and Model Armor 6.2 Responsible AI, Explainable AI, Fairness Metrics and Model Governance 6.3 Model Monitoring: Feature Skew and Drift Detection 6.4 Operational Troubleshooting, Performance Profiling and Cost Optimization 6.5 Monitoring, Testing and Evaluating Generative AI in Production

3.3 Choosing the Deployment Strategy at Design Time

Key Takeaways Explain This Topic AI

The deployment strategy is a design-time decision because it constrains model size, framework, feature availability, and preprocessing before any training begins.

The first question is online versus batch: if consumers can tolerate stale scores, batch prediction removes an entire class of serving cost and complexity.

Features that will not exist at request time cannot be used in an online model, no matter how predictive they are offline.

Serving surface choice — Agent Platform Inference, Cloud Run, GKE, or embedded/edge — follows from latency, scale, isolation, and operational ownership.

Rollout mechanics (canary, traffic split, shadow) belong to the same design decision, because they determine whether two model versions must be servable simultaneously.

Last updated: September 2026

• 3.3 Choosing the Deployment Strategy at Design Time
• Blueprint reference: Section 3.1, "Choosing the deployment strategy."

Google lists this bullet in Section 3 — scaling prototypes into models — not in Section 4, where serving lives. That placement is the lesson: deployment strategy is decided before training, because it constrains what you are allowed to build.

Why This Is a Design-Time Decision

Four things are fixed by the deployment choice and cannot be retrofitted cheaply:

Which features are legal. A feature computed from a 30-day aggregate is available in a nightly batch job. It is available online only if something maintains it in a low-latency store. If neither is true, the model cannot use it — and discovering this after training wastes the whole cycle.

How large the model may be. A model that must run on a mobile device has a hard size and memory budget. One served from an endpoint does not.

Which frameworks and versions are viable. Prebuilt inference containers cover specific framework and version pairs; anything else requires a custom container.

How preprocessing is packaged. Online serving usually requires preprocessing to travel with the model; batch scoring can apply it upstream in the pipeline.

Decision One: Online or Batch

• Question Batch Online
• Are scores needed for a known population, on a schedule? Yes —
• Do inputs only exist at request time? — Yes
• Is a few hours of staleness acceptable? Yes —
• Is per-request latency budgeted in milliseconds? — Yes
• Cost profile Resources provisioned for the run, then released Endpoint bills continuously

Batch prediction is the underused answer. Propensity scores, next-best-offer rankings, lead scores, churn risk, and content recommendations for a known user base can all be computed nightly and read from a serving table. The application then does a key lookup rather than an inference call, which is faster, cheaper, and more reliable than a live endpoint.

Online is required when the input is only knowable at request time — the contents of the current basket, the text the user just typed, the image just uploaded — or when the decision must reflect the current instant.

A common production shape is both : batch-precompute the expensive user-level features nightly, and combine them online with request-time context.

Decision Two: The Serving Surface

Surface Choose when

Agent Platform Inference endpoint Standard managed serving: autoscaling, traffic splitting, monitoring integration, accelerator support

• Batch prediction job Large offline scoring; no endpoint to maintain
• Cloud Run Lightweight CPU models, scale-to-zero, spiky traffic, an existing container-first team
• GKE Existing Kubernetes estate, complex multi-container inference graphs, fine-grained control
• BigQuery ML in place Scores consumed analytically inside the warehouse
• Edge / on-device Offline operation, strict privacy, or network-independent latency

Scale-to-zero is the distinguishing property of Cloud Run in this list. A model serving a handful of requests an hour costs nearly nothing on Cloud Run and costs a full node-hour rate on an always-on endpoint. The trade-off is cold starts, which matters if the p95 must stay low even for the first request after idleness.

Edge deployment imposes the tightest design constraints: the model must be quantized or distilled to fit, the feature set must be computable on-device, and updates require a distribution mechanism. If a scenario says a device operates without connectivity, or that images may not leave the device, edge is decided at design time and everything else follows.

Decision Three: Rollout Requirements

Rollout mechanics feed back into design because they determine whether two versions must run at once.

Traffic splitting / canary — two model versions deployed to one endpoint with a percentage split. Requires that both versions accept the same request schema.

Shadow deployment — the candidate receives a copy of production traffic and its predictions are logged but not returned. Requires the ability to mirror requests and a place to log predictions for comparison.

Blue-green — full switch between two ready deployments. Requires double capacity during the transition.

A/B test — traffic split with an outcome metric measured per arm. Requires request-level assignment and outcome tracking, which is a data-pipeline requirement, not just a serving one.

If the organization requires canary rollouts, the request and response schema must be stable across versions from the first design. A model whose input schema changes with every retrain cannot be canaried.

Feature Availability: The Most Common Design Failure

Worth stating on its own because it appears repeatedly on the exam. A model trained with a feature such as "customer's total spend over the last 90 days" will fail in online serving unless that aggregate is maintained somewhere it can be read in milliseconds — which is what Feature Store exists for. The three legal outcomes are:

Maintain the feature in an online store (Feature Store, Bigtable, Memorystore).

Compute it cheaply at request time from data already in the request.

Drop the feature and accept the accuracy cost.

Training with a feature and hoping to solve serving later is not one of them.

Exam Traps

Choosing an endpoint for a workload with no real-time requirement. Batch is cheaper.

Using an offline-only feature in an online model. Feature Store or drop it.

Ignoring cold starts when choosing scale-to-zero for latency-sensitive traffic.

Planning canary rollouts with a schema that changes per version.

Deferring the deployment decision until after training. The blueprint deliberately places it before.