Study guide
5.1 Event-Driven Messaging with Cloud Pub/Sub
Google Cloud Professional Cloud Developer Exam
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Exam and Compute Platform Decisions
1.1 Current Exam, Sources, and Scenario Method 1.2 Cloud Run Execution and Platform Selection 1.3 GKE Modes and Compute Engine Trade-offs
2 Chapter 2: Container Engineering and API Contracts
2.1 Stateless Microservices and Multi-stage Builds 2.2 Container Hardening, Signals, and Layer Caching 2.3 REST, gRPC, and Protobuf Contracts
3 Chapter 3: gRPC Compatibility and Geographic Routing
3.1 Schema Evolution and gRPC Routing 3.2 Geography, Network Tiers, and Load Balancers 3.3 Serverless NEGs, Sessions, and Edge Controls
4 Chapter 4: API Management and Caching
4.1 API Management & Rate Limiting with Apigee and API Gateway 4.2 Memorystore and Cache Access Patterns 4.3 Cache Invalidation, Capacity, and Failover
5 Chapter 5: Messaging, Task Dispatch, and Orchestration
5.1 Event-Driven Messaging with Cloud Pub/Sub 5.2 Task Queues & Rate Throttling with Cloud Tasks 5.3 Event Routing and Service Orchestration: Eventarc & Cloud Workflows
6 Chapter 6: Resilience and Relational Availability
6.1 Designing Resilient Systems: Circuit Breaking, Retries & Failover 6.2 Cloud SQL Availability and Recovery 6.3 Connection Budgets and Spanner Consistency
7 Chapter 7: Distributed Schemas and NoSQL Data Models
7.1 Spanner Keys, Indexes, and Transactions 7.2 Firestore Data, Indexes, and Transactions 7.3 Bigtable Row Keys and Replication
8 Chapter 8: Object Storage and Application Identity
8.1 Cloud Storage Lifecycle and Signed Access 8.2 Workload Identity and Service Authentication 8.3 Secret Delivery, Rotation, and IAP
9 Chapter 9: Security, Data Integration, and Developer Credentials
9.1 Private Communication, Key Rotation, and Vulnerability Response 9.2 AlloyDB Schemas and BigQuery Ingestion 9.3 CLI Configurations, Local ADC, and Cloud Shell
10 Chapter 10: Managed Development and Automated Tests
10.1 Cloud Workstations and the Container Inner Loop 10.2 Emulating Google Cloud Services & Database Auth Proxies 10.3 Automated Testing Strategies: Unit, Integration, and Mocking
11 Chapter 11: AI Development Tools and Build Pipelines
11.1 AI Coding Assistants, Context, MCP, and Test Review 11.2 Cloud Build Workers and Step Dependencies 11.3 Builders, Substitutions, Triggers, and IAM
12 Chapter 12: Artifact Security and Admission Policies
12.1 Artifact Management and Vulnerability Scanning with Artifact Registry 12.2 Build Provenance and Admission Concepts 12.3 Attestors, Policy Enforcement, and Break-glass
13 Chapter 13: Cloud Run and GKE Deployment Configuration
13.1 Cloud Run Deployment and Scaling Configuration 13.2 Cloud Run CPU, Private Egress, and Secrets 13.3 GKE Workload Controllers and Gateway Routing
14 Chapter 14: Kubernetes Resources, Health, and Progressive Releases
14.1 GKE Resources, OOM Diagnosis, and Autoscaling 14.2 Container Health Probes, Lifecycle Hooks & Graceful Shutdown 14.3 Progressive Delivery: Traffic Splitting, Canary, and Blue/Green Rollouts
15 Chapter 15: Source/Event Deployment and Cloud API Integration
15.1 Source Deployment, Event Receivers, and API Versions 15.2 Client Libraries, Service Enablement, and ADC 15.3 API Responses, Pagination, Caches, and Retries
16 Chapter 16: Tracing, Sampling, and Monitoring
16.1 Trace Context and OpenTelemetry Export 16.2 Custom Spans, Trace Analysis, and Sampling 16.3 Application Metrics, Health Checks & Alerting with Cloud Monitoring
17 Chapter 17: Structured Logs, Routing, and Error Management
17.1 Structured Logs, Trace Correlation, and Queries 17.2 Log Routing, Export Destinations, and Exclusions 17.3 Error Reporting Formats and Regression Handling
18 Chapter 18: Profiling, Generative AI, and Assisted Troubleshooting
18.1 Cloud Profiler Types and Flame Graph Analysis 18.2 Generative AI Integration with the Google Gen AI SDK 18.3 Evidence-Based Troubleshooting with AI Assistance
5.1 Event-Driven Messaging with Cloud Pub/Sub
Key Takeaways Explain This Topic AI
Cloud Pub/Sub provides scalable, asynchronous ingestion with at-least-once delivery; consumers must be idempotent to handle occasional duplicate messages.
Message ordering requires an ordering key on messages and ordering enabled on the subscription; ordered processing must preserve per-key order, with batching/redelivery behavior dependent on subscription type and client support.
Push subscriptions require publicly accessible HTTPS endpoints with valid SSL certificates and support OIDC service account authentication, while Pull subscriptions utilize synchronous pull or high-throughput StreamingPull over gRPC.
Pub/Sub natively supports direct BigQuery and Cloud Storage ingestion subscriptions, bypassing intermediate compute layers like Cloud Functions or Dataflow for raw event archival.
Subscription filters evaluate message attributes server-side, automatically acknowledging non-matching messages; delivery fees still apply, while outbound/egress delivery fees do not.
Last updated: October 2026
Event-Driven Messaging with Cloud Pub/Sub
Google Cloud Pub/Sub is a fully managed, globally distributed asynchronous messaging service designed to decouple services that produce events from services that process events. In cloud-native microservices architectures, Pub/Sub acts as the core nervous system, providing low-latency ingestion, high availability across multiple availability zones and regions, and elastic scalability from zero to millions of messages per second without manual provisioning.
Architectural Components
Pub/Sub decouples publishers and subscribers through four fundamental resources:
Topic : A named resource to which publishers send messages. Topics exist globally across Google Cloud infrastructure.
Message : The data combination transmitted through the system. Each message consists of a payload ( data , base64-encoded bytes up to 10 MB), user-defined key-value metadata pairs ( attributes ), a system-generated unique messageId , a publishTime timestamp, and an optional orderingKey .
Subscription : A named resource representing the stream of messages from a single specific topic to be delivered to the subscribing application. Multiple subscriptions can attach to a single topic, enabling 1-to-many fan-out message distribution.
Message Schemas : Formal contracts enforced by Pub/Sub that govern the structure and format of message payloads.
Message Schemas (Avro and Protocol Buffers)
To ensure data integrity between decoupled microservices, Pub/Sub supports schema enforcement using Apache Avro ( .avsc ) and Protocol Buffers ( .proto ). When a schema is attached to a topic, Pub/Sub validates published payloads before accepting them.
A topic with a schema validates message data against its configured Avro or Protobuf schema and selected JSON or binary encoding. Invalid payloads are rejected; there are no topic schema modes named FAIL_ALL_REQUESTS or ALLOW_ALL_REQUESTS . For a migration, manage compatible schema revisions and test publishers and subscribers before changing validation requirements. ## Create an Avro schema definition
- gcloud pubsub schemas create order-schema-avro \
- -type=AVRO \
--definition='{"type":"record","name":"Order","fields":[{"name":"orderId","type":"string"},{"name":"amount","type":"double"}]}'
- ## Create a topic enforcing the schema with JSON encoding
- gcloud pubsub topics create orders-topic \
- -schema=order-schema-avro \
- -message-encoding=JSON
- Subscription Types & Ingestion Modes
Pub/Sub offers four distinct subscription delivery architectures tailored to different processing models.
Subscription Type Delivery Mechanism Use Case Authentication & Delivery Notes
Pull Subscriber calls Pull or opens bidirectional StreamingPull gRPC stream Scalable backend workers on GKE, Cloud Run jobs, Compute Engine Highly efficient; supports automated lease management and batching
Push Pub/Sub initiates an HTTPS POST request to an external webhook endpoint Serverless targets (Cloud Run, Cloud Functions, App Engine) Requires valid public SSL/TLS certificate; supports Google-signed OIDC JWT authentication
BigQuery Ingestion Pub/Sub streams message payloads directly into a BigQuery table Real-time analytics, event logging, data warehousing Zero-code pipeline; supports --use-topic-schema , --drop-unknown-fields , and --write-metadata
Cloud Storage Ingestion Pub/Sub writes batches of messages directly to Cloud Storage buckets Long-term archival, backup, data lake ingestion Configurable by file batching size (e.g., 50 MB) and duration (e.g., 5 minutes) in text or Avro format
Pull Subscriptions: Synchronous vs. StreamingPull
Synchronous Pull ( projects.subscriptions.pull ) : The client application issues discrete unary RPC requests requesting up to maxMessages . This creates polling overhead and higher latency.
StreamingPull : The client opens a persistent, bidirectional gRPC streaming connection. Pub/Sub pushes messages to the client immediately as they arrive. Client libraries maintain an internal thread pool that automatically extends acknowledgment deadlines (lease management) while messages are actively being processed, providing maximum throughput and minimal delivery latency.
Push Subscriptions & OIDC Authentication
Push subscriptions deliver messages by issuing an HTTPS POST request to a webhook endpoint. The payload is delivered in the HTTP request body as a JSON object containing the base64-encoded message. ## Create a Push subscription authenticated with an OIDC Service Account
- gcloud pubsub subscriptions create orders-push-sub \
- -topic=orders-topic \
- -push-endpoint=https://orders-service-xyz.a.run.app/api/events \
- -push-auth-service-account=pubsub-invoker@my-project.iam.gserviceaccount.com \
- -ack-deadline=30
Push Acknowledgment Semantics :
Success (ACK) : The endpoint returns HTTP 102 , 200 , 201 , 202 , or 204 . Pub/Sub marks the message as acknowledged.
Failure (NACK) : The endpoint returns any 4xx/5xx status code (e.g., 429 , 500 , 503 ) or fails to respond within the ack-deadline . Pub/Sub reschedules the message for redelivery.
Acknowledgment Deadlines & Lease Extensions
When a subscriber receives a message, Pub/Sub starts an Acknowledgment Deadline timer (configurable from 10 seconds to 600 seconds , defaulting to 10 seconds).
ACK ( acknowledge ) : The subscriber finishes processing and calls acknowledge . The message is removed from the subscription queue.
NACK ( modifyAckDeadline to 0) : The subscriber encounters a transient failure and resets the deadline to 0 seconds. Pub/Sub makes the message eligible for redelivery, subject to the configured retry/backoff behavior.
Auto-Extension (Lease Management) : Modern Google Cloud client libraries automatically monitor processing time and issue modifyAckDeadline calls in the background to extend the message lease up to a configured library-specific maximum lease duration (for example, one hour). If the worker container crashes, the background thread dies, the lease expires, and Pub/Sub automatically redelivers the message to another instance.
Message Ordering & The Head-of-Line Blocking Trap
By default, Cloud Pub/Sub prioritizes throughput and availability over strict message sequence, delivering messages out of order. When an application requires strict First-In, First-Out (FIFO) processing, Message Ordering must be explicitly configured.
How Ordering Works
The subscription must have ordering enabled ( --enable-message-ordering ).
The publisher must attach a non-empty string as the ordering_key (e.g., account-1082 , device-id-9421 ) to each message.
Pub/Sub guarantees that messages with the same ordering key sent to the same region are delivered to subscribers in the exact order they were received by the service.
Warning
Ordering is per key, not a global FIFO queue. Pull responses can contain batches of messages for the same key; ordered StreamingPull callbacks are serialized per key by supporting clients. Redelivery of an earlier message can cause later messages for that key to be redelivered. Slow acknowledgments can delay progress, and dead-letter forwarding is best effort. Keep ordered handlers idempotent and monitor backlog rather than assuming exactly one outstanding message in every delivery mode. ## Enable message ordering on subscription
- gcloud pubsub subscriptions create ordered-orders-sub \
- -topic=orders-topic \
- -enable-message-ordering \
- -ack-deadline=60
- Dead-Letter Topics (DLQs)
A message that repeatedly fails processing due to formatting errors or poisoned payloads ("poison pill messages") can consume infinite retries, starving worker pools and blocking ordered streams. To mitigate this, Pub/Sub supports Dead-Letter Topics .
With the required service-agent permissions, Pub/Sub makes a best-effort attempt to forward repeatedly failing messages after approximately the configured --max-delivery-attempts (5 to 100). It forwards the unacknowledged message to the designated dead-letter topic.
Once forwarded, the message is acknowledged and removed from the primary subscription queue, unblocking subsequent messages. ## Configure a subscription with a Dead-Letter Topic
- gcloud pubsub subscriptions create orders-sub-with-dlq \
- -topic=orders-topic \
- -dead-letter-topic=orders-dlq-topic \
- -max-delivery-attempts=5 \
- -ack-deadline=30
Note
IAM Permission Requirement for DLQ :
The Cloud Pub/Sub service agent ( service-{PROJECT_NUMBER}@gcp-sa-pubsub.iam.gserviceaccount.com ) must have the roles/pubsub.subscriber role on the primary subscription and the roles/pubsub.publisher role on the dead-letter topic.
Message Attributes & Server-Side Subscription Filtering
Subscriptions can inspect message attributes and filter incoming traffic server-side using boolean expressions. Only messages matching the filter expression are delivered to subscribers. ## Create a subscription filtering on custom attributes
- gcloud pubsub subscriptions create priority-us-sub \
- -topic=orders-topic \
--message-filter='attributes.priority = "high" AND (attributes.region = "us-east" OR attributes.region = "us-west")'
- Filter Syntax Capabilities
- Exact string matching: attributes.status = "PENDING"
- Attribute existence check: attributes:category
- String prefix check: hasPrefix(attributes.category, "hardware")
Attribute values are strings; equality/inequality and prefix/existence tests are supported, not numeric range comparisons. Use a string category such as attributes.tier = "gold" .
Tip
Filtering and cost: Pub/Sub automatically acknowledges messages that do not match the subscription filter. They incur message delivery fees and applicable seek-related storage fees, but no outbound/egress message fees for delivery to the subscriber. Filtering reduces client processing and outbound traffic; it does not make all message processing free.
Fan-Out Patterns & At-Least-Once Delivery Semantics
Pub/Sub uses at-least-once delivery by default. Without a supported exactly-once pull configuration, transient network timeouts, worker crashes during acknowledgment, and distributed replication can cause duplicate message deliveries.
Achieving Consumer Idempotency
Because duplicate deliveries can occur, subscriber services must be engineered to be idempotent :
Durable business-event keys: Use a stable business identifier, not only a delivery-specific ID. Within the same database transaction, insert a unique processed-event receipt and apply the business update. Commit before acknowledging. A crash then either rolls back both operations or leaves both durable.
External side effects: Use a downstream payment API's idempotency key or a transactional outbox. Writing a Redis marker before charging risks losing the charge if the worker crashes; writing it afterward risks duplicates. A cache alone cannot atomically coordinate the effect.
Exactly-once delivery option: Supported pull subscriptions can enable regional exactly-once delivery. It prevents redelivery after a successful acknowledgment under its documented conditions, but does not deduplicate separately published business events or make external side effects atomic.
Official references checked October 18, 2026: Google Cloud 1 , Google Cloud 2 .