Study guide
14.3 Progressive Delivery: Traffic Splitting, Canary, and Blue/Green Rollouts
Google Cloud Professional Cloud Developer Exam
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Exam and Compute Platform Decisions
1.1 Current Exam, Sources, and Scenario Method 1.2 Cloud Run Execution and Platform Selection 1.3 GKE Modes and Compute Engine Trade-offs
2 Chapter 2: Container Engineering and API Contracts
2.1 Stateless Microservices and Multi-stage Builds 2.2 Container Hardening, Signals, and Layer Caching 2.3 REST, gRPC, and Protobuf Contracts
3 Chapter 3: gRPC Compatibility and Geographic Routing
3.1 Schema Evolution and gRPC Routing 3.2 Geography, Network Tiers, and Load Balancers 3.3 Serverless NEGs, Sessions, and Edge Controls
4 Chapter 4: API Management and Caching
4.1 API Management & Rate Limiting with Apigee and API Gateway 4.2 Memorystore and Cache Access Patterns 4.3 Cache Invalidation, Capacity, and Failover
5 Chapter 5: Messaging, Task Dispatch, and Orchestration
5.1 Event-Driven Messaging with Cloud Pub/Sub 5.2 Task Queues & Rate Throttling with Cloud Tasks 5.3 Event Routing and Service Orchestration: Eventarc & Cloud Workflows
6 Chapter 6: Resilience and Relational Availability
6.1 Designing Resilient Systems: Circuit Breaking, Retries & Failover 6.2 Cloud SQL Availability and Recovery 6.3 Connection Budgets and Spanner Consistency
7 Chapter 7: Distributed Schemas and NoSQL Data Models
7.1 Spanner Keys, Indexes, and Transactions 7.2 Firestore Data, Indexes, and Transactions 7.3 Bigtable Row Keys and Replication
8 Chapter 8: Object Storage and Application Identity
8.1 Cloud Storage Lifecycle and Signed Access 8.2 Workload Identity and Service Authentication 8.3 Secret Delivery, Rotation, and IAP
9 Chapter 9: Security, Data Integration, and Developer Credentials
9.1 Private Communication, Key Rotation, and Vulnerability Response 9.2 AlloyDB Schemas and BigQuery Ingestion 9.3 CLI Configurations, Local ADC, and Cloud Shell
10 Chapter 10: Managed Development and Automated Tests
10.1 Cloud Workstations and the Container Inner Loop 10.2 Emulating Google Cloud Services & Database Auth Proxies 10.3 Automated Testing Strategies: Unit, Integration, and Mocking
11 Chapter 11: AI Development Tools and Build Pipelines
11.1 AI Coding Assistants, Context, MCP, and Test Review 11.2 Cloud Build Workers and Step Dependencies 11.3 Builders, Substitutions, Triggers, and IAM
12 Chapter 12: Artifact Security and Admission Policies
12.1 Artifact Management and Vulnerability Scanning with Artifact Registry 12.2 Build Provenance and Admission Concepts 12.3 Attestors, Policy Enforcement, and Break-glass
13 Chapter 13: Cloud Run and GKE Deployment Configuration
13.1 Cloud Run Deployment and Scaling Configuration 13.2 Cloud Run CPU, Private Egress, and Secrets 13.3 GKE Workload Controllers and Gateway Routing
14 Chapter 14: Kubernetes Resources, Health, and Progressive Releases
14.1 GKE Resources, OOM Diagnosis, and Autoscaling 14.2 Container Health Probes, Lifecycle Hooks & Graceful Shutdown 14.3 Progressive Delivery: Traffic Splitting, Canary, and Blue/Green Rollouts
15 Chapter 15: Source/Event Deployment and Cloud API Integration
15.1 Source Deployment, Event Receivers, and API Versions 15.2 Client Libraries, Service Enablement, and ADC 15.3 API Responses, Pagination, Caches, and Retries
16 Chapter 16: Tracing, Sampling, and Monitoring
16.1 Trace Context and OpenTelemetry Export 16.2 Custom Spans, Trace Analysis, and Sampling 16.3 Application Metrics, Health Checks & Alerting with Cloud Monitoring
17 Chapter 17: Structured Logs, Routing, and Error Management
17.1 Structured Logs, Trace Correlation, and Queries 17.2 Log Routing, Export Destinations, and Exclusions 17.3 Error Reporting Formats and Regression Handling
18 Chapter 18: Profiling, Generative AI, and Assisted Troubleshooting
18.1 Cloud Profiler Types and Flame Graph Analysis 18.2 Generative AI Integration with the Google Gen AI SDK 18.3 Evidence-Based Troubleshooting with AI Assistance
14.3 Progressive Delivery: Traffic Splitting, Canary, and Blue/Green Rollouts
Key Takeaways Explain This Topic AI
Rolling updates incrementally replace old pods with new pods using maxSurge and maxUnavailable controls, ensuring availability during rollout but requiring continuous backward compatibility across database schemas.
Blue/Green deployments maintain two identical production environments, executing rapid cutovers via service selector or load balancer updates with fast rollback at the expense of doubling compute footprint.
Canary releases route a small fraction of production traffic to a new version, monitoring telemetry (error rates, latency) before progressively shifting 100% of traffic, drastically limiting blast radius.
Cloud Run natively supports progressive traffic splitting across immutable revisions using --to-revisions and assigns dedicated revision URL tags (--tag) for pre-release validation.
Cloud Service Mesh / Anthos Service Mesh enables fine-grained Layer 7 traffic routing, weighted traffic shifting, header-based routing, and fault injection testing using VirtualService and DestinationRule CRDs.
Last updated: October 2026
Progressive Delivery: Traffic Splitting, Canary, and Blue/Green Rollouts
Releasing software into production environments introduces inherent operational risk. Even with rigorous automated testing and CI pipelines, subtle bugs, performance regressions, or database contention issues can escape into production. Progressive Delivery combines continuous delivery with advanced deployment strategies, traffic shifting, and real-time observability to expose new software versions to a small audience before full-scale release, minimizing the "blast radius" of potential defects.
For the Google Cloud Professional Cloud Developer exam, you must master the mechanics of Rolling updates, Blue/Green deployments, and Canary rollouts across both Google Kubernetes Engine and Cloud Run, including Cloud Service Mesh Layer 7 routing and automated rollback patterns.
Deployment Release Strategies: Rolling, Blue/Green & Canary
Choosing the correct deployment strategy involves balancing deployment speed, infrastructure cost, rollback complexity, and database schema compatibility:
Deployment Strategy Resource Overhead Rollback Latency Database Compatibility Requirement Traffic Splitting Mechanism
Rolling Update Minimal (+25% surge) Slow (requires rolling back pod by pod) Strictly backward-compatible (Versions N and N+1 run concurrently) Random round-robin based on replica ratios
Blue/Green High (+100% compute) Rapid (routing update with propagation and connection-draining considerations) Compatible schema/data evolution and a tested rollback plan Routing/selector update; propagation and existing connections still matter
Canary Release Low (+5% to +10%) Fast (shift traffic back to baseline) Strictly backward-compatible (Canary handles live production data) Weighted Layer 7 traffic splitting (5% canary, 95% baseline)
Kubernetes Rolling Updates: maxSurge & maxUnavailable
The default Kubernetes Deployment strategy is the RollingUpdate . It replaces old pods with new pods incrementally according to two parameters declared in spec.strategy.rollingUpdate :
maxSurge : The maximum number of pods that can be created above the desired replica count. Can be an absolute number (e.g., 2 ) or a percentage (e.g., 25% ).
maxUnavailable : The maximum number of pods that can be unavailable during the update process relative to the desired replica count. spec:
replicas: 20
strategy:
type: RollingUpdate
rollingUpdate:
- # Allows up to 25 pods during rollout (20 + 5)
- maxSurge: 25%
- # Ensures at least 20 pods remain available at all times (zero dropped capacity)
- maxUnavailable: 0
Exam Trap: During a Rolling Update, pods running version N and pods running version N+1 exist simultaneously in the cluster, processing requests from the same database. Any database schema modifications MUST be backward and forward compatible (using the Expand-Contract / Parallel Run design pattern). If a database migration drops a column required by version N, rolling updates will fail catastrophically.
Blue/Green Deployments
In a Blue/Green deployment , two identical environments exist:
Blue: Current production environment serving 100% of live traffic.
Green: Target environment running the newly deployed application version. 1. Blue is active (100% live user traffic).
2. Deploy version 2 to Green environment.
3. Run integration smoke tests against Green's private test endpoint.
4. Update Service label selector or Load Balancer backend from Blue to Green.
5. Traffic changes propagate asynchronously; existing connections and DNS caches can keep using old paths.
6. If errors occur, by changing routing revert the Service selector back to Blue.
- In Kubernetes, the cutover is executed by modifying the Service's spec.selector : ## Request cutover to Green
- kubectl patch service order-service -p '{"spec":{"selector":{"version":"green"}}}'
- Canary Deployments
A Canary deployment exposes a new version to a small, controlled percentage of real user traffic (e.g., 5%) while 95% of traffic remains on the stable baseline. The canary version's telemetry—error rates, HTTP 5xx responses, transaction latencies, and Cloud Logging exceptions—is monitored continuously. If telemetry remains healthy, traffic is incrementally shifted (5% -> 25% -> 50% -> 100%). If anomalies occur, traffic can be routed back back to 100% baseline, impacting only a tiny fraction of requests.
Cloud Run Traffic Management & Tagged Revisions
Cloud Run provides built-in, serverless progressive delivery capabilities without requiring a service mesh or external load balancer.
Revision Tagging ( --tag )
Every deployment to Cloud Run creates an immutable Revision . By applying traffic tags, developers can assign dedicated, deterministic URLs to specific revisions without assigning service-URL traffic to them; tag URLs still obey the service's IAM and ingress settings: ## Step 1: Deploy new revision with a 'canary' tag and 0% public traffic
- gcloud run deploy payment-service \
- -image=us-central1-docker.pkg.dev/my-project/app-repo/payment-service:v2.0.0 \
- -region=us-central1 \
- -no-traffic \
- -tag=canary
- Cloud Run provisions a dedicated URL for the revision: https://canary---payment-service-67abc-uc.a.run.app
Automated integration test suites can target this tagged URL directly to validate configuration using isolated test records and explicitly approved access; a tagged revision still shares backing resources unless separated before routing real end-user traffic.
Percentage-Based Traffic Splitting
Once pre-release verification succeeds, developers split traffic between the stable production revision and the canary revision using --to-revisions : ## Step 2: Route 90% to stable revision and 10% to canary revision
- gcloud run services update-traffic payment-service \
- -region=us-central1 \
- -to-revisions=payment-service-00001-v1=90,payment-service-00002-v2=10
- Rapid Zero-Downtime Rollback
If the canary revision introduces unexpected latency or exceptions, rolling back does not require rebuilding or redeploying container images. Because previous revisions remain available for routing (but may need new instances), routing 100% of traffic back to the previous revision takes effect in the control plane's propagation interval: ## Instant rollback to known-good revision
- gcloud run services update-traffic payment-service \
- -region=us-central1 \
- -to-revisions=payment-service-00001-v1=100
Alternatively, to always route 100% of traffic to the latest successfully deployed revision: gcloud run services update-traffic payment-service \
- -region=us-central1 \
- -to-latest
- Advanced Layer 7 Routing with Cloud Service Mesh / Anthos Service Mesh
For microservices running on GKE, Cloud Service Mesh (managed Anthos Service Mesh based on open-source Istio) provides Layer 7 intelligent traffic routing via Envoy sidecar proxies deployed alongside application containers.
Cloud Service Mesh decouples traffic routing policies from application code using two primary Custom Resource Definitions (CRDs):
VirtualService: Defines routing rules, URI path matching, header matching, and destination weight splits.
DestinationRule: Defines policies applied to traffic once routed, including load balancing algorithms (round robin, least connection), connection pools, and pod subset labels.
- Weight-Based Canary Routing Manifest apiVersion: networking.istio.io/v1alpha3
- kind: DestinationRule
metadata:
name: order-service-dest
spec:
host: order-service
subsets:
- name: v1-stable
labels:
- version: v1
- name: v2-canary
labels:
- version: v2
- --
- apiVersion: networking.istio.io/v1alpha3
- kind: VirtualService
metadata:
name: order-service-vs
spec:
hosts:
- order-service
http:
- route:
- destination:
- host: order-service
- subset: v1-stable
- weight: 95
- destination:
- host: order-service
- subset: v2-canary
- weight: 5
- Header-Based & Cookie-Based Routing
Cloud Service Mesh allows routing requests based on HTTP headers, enabling selected cohorts identified through a trusted gateway or authenticated application logic to test new versions in production without impacting general users: apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: order-service-vs
spec:
hosts:
- order-service
http:
# Rule 1: Route internal QA / Beta testers to canary based on header
- match:
- headers:
x-release-channel:
exact: canary
route:
- destination:
- host: order-service
- subset: v2-canary
- # Rule 2: Default rule for all other users routes to stable baseline
- route:
- destination:
- host: order-service
- subset: v1-stable
- Fault Injection: Chaos Engineering for Microservices
Before releasing a new revision, developers can use Cloud Service Mesh to inject artificial faults into traffic destined for the service, validating that upstream calling services properly handle timeouts and circuit breaking: apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: payment-service-fault
spec:
hosts:
- payment-service
http:
- fault:
# Inject artificial 5-second delay into 10% of requests
delay:
percentage:
- value: 10.0
- fixedDelay: 5s
- # Inject HTTP 503 Service Unavailable into 5% of requests
abort:
percentage:
- value: 5.0
- httpStatus: 503
route:
- destination:
- host: payment-service
- subset: v2-canary
- Automated Rollback Strategies with Cloud Monitoring
In enterprise progressive delivery pipelines, human operators should not manually watch dashboards during a canary release. Instead, deployment pipelines integrate with Cloud Monitoring alerting policies to implement deliberate automated rollback logic; Cloud Monitoring alone does not execute gcloud commands:
Baseline Telemetry Capture: Capture 5-minute pre-release baseline error rate ( loadbalancing.googleapis.com/https/request_count filtered by response code class 5XX ) and latency P99.
Canary Deployment: Shift 5% traffic to canary revision via Cloud Run or Cloud Service Mesh.
Automated Evaluation Gate: Execute a Cloud Build or Cloud Deploy verification step querying Cloud Monitoring metrics for the canary revision over a 10-minute window.
Trigger Rollback on SLO Violation: If the canary error rate exceeds 1% or latency P99 increases by more than 20% , an automated webhook triggers gcloud run services update-traffic --to-revisions=[STABLE_REV]=100 , removing the canary's service-URL traffic after routing propagation; tag access and in-flight requests are separate.