Study guide
8.3 Secret Delivery, Rotation, and IAP
Google Cloud Professional Cloud Developer Exam
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Exam and Compute Platform Decisions
1.1 Current Exam, Sources, and Scenario Method 1.2 Cloud Run Execution and Platform Selection 1.3 GKE Modes and Compute Engine Trade-offs
2 Chapter 2: Container Engineering and API Contracts
2.1 Stateless Microservices and Multi-stage Builds 2.2 Container Hardening, Signals, and Layer Caching 2.3 REST, gRPC, and Protobuf Contracts
3 Chapter 3: gRPC Compatibility and Geographic Routing
3.1 Schema Evolution and gRPC Routing 3.2 Geography, Network Tiers, and Load Balancers 3.3 Serverless NEGs, Sessions, and Edge Controls
4 Chapter 4: API Management and Caching
4.1 API Management & Rate Limiting with Apigee and API Gateway 4.2 Memorystore and Cache Access Patterns 4.3 Cache Invalidation, Capacity, and Failover
5 Chapter 5: Messaging, Task Dispatch, and Orchestration
5.1 Event-Driven Messaging with Cloud Pub/Sub 5.2 Task Queues & Rate Throttling with Cloud Tasks 5.3 Event Routing and Service Orchestration: Eventarc & Cloud Workflows
6 Chapter 6: Resilience and Relational Availability
6.1 Designing Resilient Systems: Circuit Breaking, Retries & Failover 6.2 Cloud SQL Availability and Recovery 6.3 Connection Budgets and Spanner Consistency
7 Chapter 7: Distributed Schemas and NoSQL Data Models
7.1 Spanner Keys, Indexes, and Transactions 7.2 Firestore Data, Indexes, and Transactions 7.3 Bigtable Row Keys and Replication
8 Chapter 8: Object Storage and Application Identity
8.1 Cloud Storage Lifecycle and Signed Access 8.2 Workload Identity and Service Authentication 8.3 Secret Delivery, Rotation, and IAP
9 Chapter 9: Security, Data Integration, and Developer Credentials
9.1 Private Communication, Key Rotation, and Vulnerability Response 9.2 AlloyDB Schemas and BigQuery Ingestion 9.3 CLI Configurations, Local ADC, and Cloud Shell
10 Chapter 10: Managed Development and Automated Tests
10.1 Cloud Workstations and the Container Inner Loop 10.2 Emulating Google Cloud Services & Database Auth Proxies 10.3 Automated Testing Strategies: Unit, Integration, and Mocking
11 Chapter 11: AI Development Tools and Build Pipelines
11.1 AI Coding Assistants, Context, MCP, and Test Review 11.2 Cloud Build Workers and Step Dependencies 11.3 Builders, Substitutions, Triggers, and IAM
12 Chapter 12: Artifact Security and Admission Policies
12.1 Artifact Management and Vulnerability Scanning with Artifact Registry 12.2 Build Provenance and Admission Concepts 12.3 Attestors, Policy Enforcement, and Break-glass
13 Chapter 13: Cloud Run and GKE Deployment Configuration
13.1 Cloud Run Deployment and Scaling Configuration 13.2 Cloud Run CPU, Private Egress, and Secrets 13.3 GKE Workload Controllers and Gateway Routing
14 Chapter 14: Kubernetes Resources, Health, and Progressive Releases
14.1 GKE Resources, OOM Diagnosis, and Autoscaling 14.2 Container Health Probes, Lifecycle Hooks & Graceful Shutdown 14.3 Progressive Delivery: Traffic Splitting, Canary, and Blue/Green Rollouts
15 Chapter 15: Source/Event Deployment and Cloud API Integration
15.1 Source Deployment, Event Receivers, and API Versions 15.2 Client Libraries, Service Enablement, and ADC 15.3 API Responses, Pagination, Caches, and Retries
16 Chapter 16: Tracing, Sampling, and Monitoring
16.1 Trace Context and OpenTelemetry Export 16.2 Custom Spans, Trace Analysis, and Sampling 16.3 Application Metrics, Health Checks & Alerting with Cloud Monitoring
17 Chapter 17: Structured Logs, Routing, and Error Management
17.1 Structured Logs, Trace Correlation, and Queries 17.2 Log Routing, Export Destinations, and Exclusions 17.3 Error Reporting Formats and Regression Handling
18 Chapter 18: Profiling, Generative AI, and Assisted Troubleshooting
18.1 Cloud Profiler Types and Flame Graph Analysis 18.2 Generative AI Integration with the Google Gen AI SDK 18.3 Evidence-Based Troubleshooting with AI Assistance
8.3 Secret Delivery, Rotation, and IAP
Key Takeaways Explain This Topic AI
Cloud Run environment secrets resolve at instance startup; mounted latest values resolve on access.
A rotation schedule sends notifications; implemented automation changes credentials and verifies adoption.
Do not trust plain identity headers when the signed IAP assertion supplies verifiable claims.
Last updated: October 2026
Secret Manager: Architecture, Access & Delivery Patterns
Google Cloud Secret Manager provides centralized, audited, and encrypted storage for API keys, database credentials, TLS certificates, and sensitive configuration strings.
Key Concepts
Secrets and Versions: A Secret is a global resource that holds metadata and rotation policies. The actual sensitive payload resides inside immutable Secret Versions (e.g., projects/my-project/secrets/db-pass/versions/1 ).
Access Control: Enforce least privilege using granular IAM roles:
roles/secretmanager.secretAccessor : Permits fetching the secret payload ( secretmanager.versions.access ). This is the only role application service accounts should receive.
roles/secretmanager.viewer : Permits reading secret metadata only (names, versions, labels); cannot view secret payloads.
roles/secretmanager.admin : Grants full administrative control.
Secret Delivery Patterns: Dynamic SDK vs. Env Vars vs. Volume Mounts
Delivery Pattern Deployment Method Rotation Handling Security & Memory Profile
Dynamic SDK Retrieval Code invokes Secret Manager Client Library directly Instant (Application can refresh secret payload on-demand or with TTL cache) The application can keep it in memory; logging, dump and persistence behavior remains under application control.
Environment Variable Injection Cloud Run --set-secrets="DB_PASS=db-secret:latest" Requires Redeployment/Restart ; running instances keep stale value Visible in process inspect ( /proc/*/environ ); vulnerable to log dumps.
Volume Mounts Cloud Run --set-secrets="/secrets/db=db-secret:latest" or GKE CSI Driver Cloud Run reads resolve the selected version on access; GKE CSI refresh behavior must be configured and checked Backed by in-memory tmpfs; application reads file from disk when refreshing.
Exam Trap: When secrets are injected into Cloud Run as environment variables using :latest , Cloud Run evaluates the secret version only when the container instance starts . If a developer adds a new secret version in Secret Manager, running container instances continue using the old secret value until they are restarted or redeployed. To achieve automated rotation without restarts, mount the secret as a volume or retrieve it dynamically using the Client Library .
Automated Secret Rotation Patterns
Secret Manager supports automated rotation schedules configured via Cloud Pub/Sub and Cloud Functions (or Cloud Run jobs): +--------------------------------------------------------------------------+
- | Automated Secret Rotation Architecture |
- +--------------------------------------------------------------------------+
- | |
- | [ Secret Manager ] |
- | | |
- | | 1. Rotation Schedule Timer Fires (e.g., every 30 days) |
- | v |
- | [ Cloud Pub/Sub Topic ] ('projects/my-project/topics/secret-rotate') |
- | | |
- | | 2. Delivers Rotation Notification Event |
- | v |
- | [ Cloud Function: Rotator ] |
- | | |
- | +---> 3. Generates New Secure Password / API Key |
- | +---> 4. Updates Downstream Service (e.g., ALTER USER in Cloud SQL|
- | +---> 5. Adds New Secret Version in Secret Manager |
- | +---> 6. Verifies Adoption; Retires Old Version after Rollback Window |
- +--------------------------------------------------------------------------+
- Zero-Downtime Dual-Version Strategy
When rotating database passwords or API keys, downstream systems must support a dual-credential grace period :
The rotation Cloud Function creates a new database user or secondary API key.
The function adds the new credential as a new version in Secret Manager.
Microservices pick up the new secret version.
Once all clients are verified to have transitioned, including slow workers and connection pools (a fixed one-hour delay is not proof), the old credential version is disabled and destroyed.
Identity-Aware Proxy (IAP) & Zero-Trust Validation
Identity-Aware Proxy (IAP) establishes a zero-trust perimeter around web applications deployed on App Engine, Compute Engine, GKE, or Cloud Run. It verifies user identity and context (device health, geographic location, IP address) at the Google Cloud Load Balancing edge before permitting traffic to reach backend services. +--------------------------------------------------------------------------+
- | Identity-Aware Proxy (IAP) Zero-Trust Ingress Flow |
- +--------------------------------------------------------------------------+
- | [ Corporate User ] |
- | | |
- | | 1. HTTPS GET /dashboard |
- | v |
- | [ Cloud Load Balancer + Identity-Aware Proxy (IAP) ] |
- | | |
- | | 2. Evaluates Google Identity & BeyondCorp Context Policies |
- | | (Checks roles/iap.httpsResourceAccessor) |
- | | 3. Injects Cryptographically Signed Assertion Header: |
- | | 'X-Goog-IAP-JWT-Assertion' |
- | v |
- | [ Backend Application (GKE / Cloud Run) ] |
- | | |
- | | 4. Backend Cryptographically Validates JWT Signature via JWKS |
- | | 5. Verifies 'aud' claim: /projects/NUM/global/backendServices/ID|
- | | 6. Extracts verified user identity: claims['email'] |
- +--------------------------------------------------------------------------+
- The Developer Trap: Plain Header Spoofing
When IAP successfully authenticates a user, it injects plain HTTP headers into the proxied request:
- X-Goog-Authenticated-User-Email
- X-Goog-Authenticated-User-Id
Exam Trap: Never rely solely on plain HTTP headers like X-Goog-Authenticated-User-Email for authorization. If an attacker gains access to the internal VPC or compromises another service on the same network, they can forge plain HTTP headers and bypass IAP authentication. To guarantee authenticity, the backend application must cryptographically validate the signed JWT assertion header.
Validating X-Goog-IAP-JWT-Assertion in Code
IAP injects a signed JSON Web Token (JWT) in the X-Goog-IAP-JWT-Assertion header. Secure backend applications must perform four validation steps:
Fetch Public Keys: Retrieve Google's public JSON Web Key Set (JWKS) from https://www.gstatic.com/iap/verify/public_key-jwk .
Verify Cryptographic Signature: Verify the JWT's ECDSA signature using the public keys.
Validate Issuer ( iss ): Confirm that the issuer claim equals https://cloud.google.com/iap .
Validate Audience ( aud ): Confirm that the audience claim matches your specific backend service string: /projects/PROJECT_NUMBER/global/backendServices/SERVICE_ID ## Python Flask IAP JWT Cryptographic Verification
- from google.auth.transport import requests
- from google.oauth2 import id_token
def validate_iap_jwt(iap_jwt, expected_audience):
try:
- # Decodes and verifies signature against Google's public keys
- decoded_jwt = id_token.verify_token(
- iap_jwt,
- requests.Request(),
- audience=expected_audience,
- certs_url='https://www.gstatic.com/iap/verify/public_key-jwk'
- )
if decoded_jwt.get('iss') != 'https://cloud.google.com/iap':
- raise PermissionError('Unexpected IAP issuer')
- return decoded_jwt['email'], decoded_jwt['sub']
except Exception as e:
- raise PermissionError(f"IAP JWT verification failed: {e}")
- Application Security & Identity Architecture Matrix
Security Mechanism Identity Binding Primary Attack Vector Mitigated Performance & Scaling Considerations
GKE Workload Identity KSA -> GSA via Pod Metadata Server Credential exfiltration; static JSON key theft Metadata caching eliminates latency; native token refreshing.
Cloud Run Service Identity GSA attached via --service-account Uncontrolled service-to-service privilege escalation Integrated into container startup; short-lived tokens in memory.
Secret Manager Volume Mount In-memory tmpfs file Stale credentials across long-lived container instances Cloud Run resolves file reads; application caches and GKE CSI rotation need explicit handling.
Identity-Aware Proxy (IAP) Google Cloud Identity / BeyondCorp Direct backend network exposure; credential stuffing Edge authentication reduces backend load; requires JWT verification.