Study guide
4.1 Cloud Pub/Sub Core Architecture and Ingestion
Google Cloud Professional Data Engineer Exam
Track your progress as you read
Finished sections get a check mark.
1 Introduction & Exam Overview
0.1 About the Google Cloud Professional Data Engineer Exam 0.2 Study Strategy & Cloud Data Engineering Methodology
2 Chapter 1: Designing for Security and Compliance
1.1 IAM Roles, Service Accounts, and Least Privilege for Data 1.2 Data Encryption, Key Management, and Privacy with Cloud DLP 1.3 Compliance, Data Sovereignty, and Geographic Placement 1.4 Multi-Environment Architecture: Development, Staging, and Production
3 Chapter 2: Designing for Reliability and Resilience
2.1 High Availability, Disaster Recovery, and Failover Patterns 2.2 Fault Tolerance and ACID Guarantees in Distributed Systems 2.3 Data Validation, Cleansing, and Quality Assurance with Dataform 2.4 AI-Assisted Data Preparation and LLM Query Generation
4 Chapter 3: Architecture Patterns and Flexibility
3.1 Modern Data Architectures: Data Lake, Warehouse, Lakehouse, and Mesh 3.2 Data Migration Strategies and Hybrid/Multi-Cloud Integration 3.3 Database Migration Service and Heterogeneous Database Migrations
5 Chapter 4: Ingestion Architectures: Pub/Sub and Streaming Sources
4.1 Cloud Pub/Sub Core Architecture and Ingestion 4.2 Pub/Sub Advanced Patterns: Dead-Lettering, Ordering, and Schemas 4.3 Change Data Capture (CDC) with Datastream and Batch Ingestion
6 Chapter 5: Stream and Batch Processing with Apache Beam and Cloud Dataflow
5.1 Apache Beam Core Concepts and Pipeline Design 5.2 Advanced Windowing, Watermarks, and Triggers 5.3 Dataflow Pipeline Optimization, Scaling, and Templates
7 Chapter 6: Data Transformation with Dataproc, Data Fusion, and Workflow Orchestration
6.1 Apache Spark and Hadoop with Cloud Dataproc 6.2 Visual Data Pipelines with Cloud Data Fusion 6.3 Workflow Orchestration with Cloud Composer 6.4 AI Data Enrichment inside Data Pipelines
8 Chapter 7: Storage Service Selection and Cloud Storage Architecture
7.1 Google Cloud Storage Service Selection Decision Matrix 7.2 Cloud Storage Architecture, Classes, and Lifecycle Management
9 Chapter 8: Deep Dive into Google BigQuery Storage
8.1 BigQuery Storage Architecture and Data Organization 8.2 Partitioning and Clustering for Performance and Cost 8.3 BigLake and Hybrid Querying across Clouds 8.4 Data Warehouse Modeling: Star Schemas, Denormalization, and Slowly Changing Dimensions
10 Chapter 9: Operational and NoSQL Databases: Bigtable and Cloud Spanner
9.1 Cloud Bigtable Architecture and Row Key Design 9.2 Cloud Bigtable Schema Management and Performance Tuning 9.3 Cloud Spanner: Globally Scalable Relational Database
11 Chapter 10: BigQuery Query Performance and Optimization
10.1 BigQuery SQL Tuning and Performance Best Practices 10.2 BigQuery BI Engine and Materialized Views 10.3 Data Sharing and Analytics Hub 10.4 Preparing Data for Visualization and BI Tool Connectivity
12 Chapter 11: Machine Learning Readiness and Data Governance
11.1 Machine Learning with BigQuery ML 11.2 Feature Engineering and Data Preparation for AI/ML 11.3 Data Governance, Cataloging, and Discovery with Dataplex 11.4 Unstructured Data, Embeddings, and Retrieval-Augmented Generation
13 Chapter 12: Site Reliability Engineering and Monitoring for Data Systems
12.1 SRE Principles for Data: SLIs, SLOs, and SLAs 12.2 Observability: Cloud Monitoring, Cloud Logging, and Alerting 12.3 Troubleshooting Data Pipelines and Workload Failures
14 Chapter 13: Workload Automation, CI/CD, and Cost Optimization
13.1 CI/CD and Version Control for Data Pipelines 13.2 BigQuery Cost Management and Capacity Planning 13.3 Holistic Cost Optimization across GCP Data Services 13.4 Query Workload Management: Interactive vs. Batch Jobs, Queues, and Quotas
4.1 Cloud Pub/Sub Core Architecture and Ingestion
Key Takeaways Explain This Topic AI
Cloud Pub/Sub provides a globally distributed, serverless messaging architecture that fully decouples event producers from downstream consumers using topics and independent subscriptions.
StreamingPull relies on persistent bidirectional gRPC streams to achieve high throughput and automated ack deadline extension, whereas Synchronous Pull uses unary RPCs best suited for batch or micro-batch workloads.
Push subscriptions transmit messages as HTTPS POST requests with Google-signed OpenID Connect (OIDC) JWT bearer tokens, securing webhooks on Cloud Run and Cloud Functions without public exposure.
Direct Export subscriptions deliver messages directly to BigQuery tables or Cloud Storage buckets without provisioning intermediate compute instances like Dataflow or Cloud Functions.
The Seek feature allows rewinding subscription cursors to historical timestamps or named snapshots for disaster recovery, while ordering keys guarantee strict per-key sequential processing at global scale.
Last updated: September 2026
4.1 Cloud Pub/Sub Core Architecture and Ingestion
Important
On the Google Cloud Professional Data Engineer exam, Cloud Pub/Sub serves as the primary ingestion backbone for real-time streaming architectures. Key exam scenarios test your ability to select the correct subscription type (StreamingPull vs. Push vs. Direct Export), enforce secure webhook authentication via OpenID Connect (OIDC), configure message retention for zero-data-loss replay, and achieve deterministic message ordering without crippling ingestion throughput.
Modern enterprise data platforms must ingest high-velocity, heterogeneous event streams from distributed sources—including Internet of Things (IoT) sensor fleets, mobile application clickstreams, e-commerce checkout events, transactional database mutations, and microservice audit logs. These ingestion architectures must absorb massive traffic spikes without dropping records, introduce zero tight coupling between producers and consumers, and scale elastically across global geographical footprints.
Google Cloud Pub/Sub addresses these requirements through a fully managed, serverless publish/subscribe messaging infrastructure designed to process hundreds of millions of messages per second with single-digit millisecond latency. Understanding its internal topology, subscription mechanisms, delivery semantics, and operational controls is essential for designing resilient data pipelines on Google Cloud.
Distributed Messaging Architecture: Topics, Subscriptions, and Messages
Cloud Pub/Sub decouples systems that produce events from systems that process events. Unlike traditional on-premises message brokers (such as RabbitMQ or active/passive JMS queues) that bind queues directly to worker processes and require manual cluster sizing, Pub/Sub separates ingestion endpoints from message consumption channels through three core primitives: +--------------------+ +--------------------+ +------------------------+
- | Event Producer | ------> | Pub/Sub Topic | ------> | Subscription A (Pull) | ---> Dataflow Pipeline
- | (IoT / App Log) | Publish | (Global Ingestion) | Routing +------------------------+
- +--------------------+ +--------------------+ | +------------------------+
- +--> | Subscription B (Push) | ---> Cloud Run Webhook
- +------------------------+
- +------------------------+
- | Subscription C (Direct)| ---> BigQuery Table
- +------------------------+
- 1. Topics
A Topic is a named, globally accessible resource to which publishers send messages. Topics do not store messages independently; instead, a topic acts as a logical ingestion gateway that instantly routes published messages across all attached subscriptions. Topics automatically distribute traffic across Google Cloud's global network of forwarder and router servers, scaling horizontally without manual partition provisioning, broker rebalancing, or cluster capacity management.
When a message is published, the Pub/Sub forwarder layer persists the payload redundantly across multiple availability zones within the target region (or multi-region) using Google's distributed Colossus storage filesystem before acknowledging receipt to the publisher. This guarantees immediate multi-zone durability and prevents data loss in the event of an infrastructure zone outage.
2. Subscriptions
A Subscription represents an independent, stateful stream of messages originating from a single topic. Multiple subscriptions can attach to the same topic, implementing the fan-out pattern where each subscriber group receives its own complete, isolated copy of the message stream. For example, a single orders-v1 topic can fan out simultaneously to a real-time fraud detection Dataflow pipeline, a Cloud Storage archival export subscription, and a Cloud Run notifications webhook.
Conversely, multiple worker instances reading from the same subscription implement the competing consumers pattern , automatically load-balancing message processing across all active workers. Pub/Sub dynamically assigns message leases among active worker threads, ensuring that no two workers concurrently process the same message under normal operation.
Server-Side Message Filtering
Subscriptions support server-side message filtering . By specifying a filter expression using Pub/Sub attribute syntax, the Pub/Sub routing plane evaluates and drops non-matching messages before dispatch: attributes.event_type = "PAYMENT_SETTLED" AND attributes.currency = "USD"
Key architectural characteristics of server-side filtering on the exam include:
Bandwidth and Compute Optimization: Filtering occurs entirely within the Pub/Sub service plane. Non-matching messages are automatically acknowledged by the system without being transmitted over the network to the consumer, saving egress bandwidth and worker CPU cycles.
Filterable Fields: Filters operate on message attributes (e.g., attributes.key = "value" , hasPrefix(attributes.key, "prefix") , or boolean checks) and certain system attributes (such as attributes.type ). Message payloads cannot be inspected for filtering.
Billing Impact: Filtered-out messages are billed as standard subscription delivery throughput. However, downstream egress fees and consumer compute costs are completely avoided.
3. Messages
A Pub/Sub Message contains the following components:
Payload ( data ): The core message body, up to 10 MB in size, represented as a base64-encoded byte array. Payloads commonly contain serialized JSON, Apache Avro, or Protocol Buffer records.
Attributes ( attributes ): An optional key-value dictionary of string pairs (up to 100 attributes per message, maximum 1,024 bytes per key/value pair). Attributes convey routing metadata, tenant identifiers, schema versions, or filtering criteria.
Message ID ( messageId ): A globally unique string assigned by the Pub/Sub service upon successful publish confirmation.
Publish Timestamp ( publishTime ): A server-assigned UTC timestamp marking the exact millisecond when the message was persisted and acknowledged to the publisher.
Ordering Key ( orderingKey ): An optional string identifier used to enforce FIFO delivery among messages sharing the same key.
- Message Component Data Type Maximum Size / Limit Architectural Purpose
- data (Payload) bytes (Base64) 10 MB Raw business payload (JSON, Avro, Protobuf, binary)
attributes map<string, string> 100 pairs; 1,024 bytes/pair Routing metadata, server-side filtering tags, tenant IDs
- messageId string Unique 64-bit int string Server-assigned identifier for tracing and deduplication
- publishTime timestamp (UTC) Microsecond precision Event publish time assigned upon multi-zone storage commit
- orderingKey string 1,024 bytes Partition key for strict in-order message dispatch
- Subscription Types and Delivery Models
Choosing between Pull, Push, and Direct Export delivery models is a central architectural decision evaluated on the certification exam. The decision depends entirely on consumer architecture, throughput requirements, network topology, and processing latency SLAs. +-----------------------------------------------------+
- | Cloud Pub/Sub Subscription |
- +-----------------------------------------------------+
- | | |
- [StreamingPull / Sync Pull] [HTTPS POST Push] [Direct Export Push]
- | | |
- v v v
- +---------------------+ +-----------------+ +-----------------+
- | Consumer Workers | | HTTPS Endpoint | | Managed Sink |
- | (Dataflow / GKE) | | (Cloud Run / App| | (BigQuery / GCS)|
- | Pulls Messages | | Receives Push | | Direct Storage |
- +---------------------+ +-----------------+ +-----------------+
- 1. Pull Subscriptions
In a pull subscription, the subscribing client initiates all communication with the Pub/Sub service by requesting messages over Remote Procedure Calls (RPC):
A. StreamingPull (Standard for Real-Time Stream Ingestion)
Mechanism: Established via a persistent, bidirectional gRPC stream ( StreamingPullRequest and StreamingPullResponse ) operating over HTTP/2 multiplexed TCP connections.
Throughput & Latency: The client opens persistent channels over which the Pub/Sub server pushes batches of messages the instant they are published, yielding sub-second delivery latency with zero polling overhead.
Automated Lease Management: Modern Google Cloud client libraries automatically maintain background heartbeat threads. While a worker thread is executing, the client library automatically issues modifyAckDeadline requests in the background, extending the message's acknowledgement deadline dynamically until processing finishes.
Recommended Use Cases: High-throughput streaming workloads requiring hundreds of megabytes per second, including Cloud Dataflow (Apache Beam) , Apache Spark on Cloud Dataproc , and large-scale Google Kubernetes Engine (GKE) microservice fleets.
B. Synchronous Pull (Unary Pull)
Mechanism: Relies on standard unary RPC requests ( PullRequest ), where the client requests up to maxMessages and blocks until messages arrive or a timeout occurs.
Throughput & Latency: High polling latency and connection establishment overhead. If the subscription backlog is empty, workers spend compute cycles issuing empty pull requests.
Recommended Use Cases: Periodic batch jobs, Cloud Run scheduled jobs, lightweight administrative scripts, or legacy applications that process discrete micro-batches on cron schedules.
2. Push Subscriptions
In a push subscription, Cloud Pub/Sub acts as an HTTP client, delivering messages by sending HTTPS POST requests containing JSON-wrapped message envelopes to an HTTPS webhook endpoint: {
- "message": {
- "attributes": {
- "event_type": "ORDER_PLACED"
- },
- "data": "eyJvcmRlcl9pZCI6ICJPUkQtODkwMSIsICJhbW91bnQiOiAxNDkuOTV9",
- "messageId": "1204981290381029",
- "publishTime": "2026-09-14T17:00:00.123456Z"
- },
- "subscription": "projects/my-proj/subscriptions/order-webhook-sub"
- }
- Endpoint Authentication via OpenID Connect (OIDC)
To protect webhook endpoints from unauthorized public invocations, push subscriptions integrate natively with Google Cloud IAM using OpenID Connect (OIDC) :
Configuration: The push subscription is configured with a dedicated user-managed service account ( serviceAccountEmail ) and an optional audience string ( audience ).
Token Generation: Pub/Sub generates a Google-signed JWT bearer token placed in the HTTP Authorization: Bearer <JWT_TOKEN> header of every outbound POST request.
Validation at Ingress: The receiving service (such as Cloud Run or Cloud Functions) verifies the signature against Google's public keys. Cloud Run automatically enforces this when ingress authentication is enabled, checking that the invoking service account possesses the roles/run.invoker permission.
Acknowledgement Semantics and HTTP Status Codes
Success ( 200 , 201 , 202 , 204 ): Pub/Sub considers the message delivered and marks it acknowledged ( ACK ).
Transient Failure ( 5xx , network timeout, or retryable 4xx ): Pub/Sub considers delivery failed ( NACK ) and reschedules the message for redelivery according to the subscription retry policy.
Permanent Dropped ( 404 Not Found , 410 Gone ): Pub/Sub marks the message acknowledged and drops it permanently, preventing infinite retries against deleted endpoints.
3. Direct Export Subscriptions (BigQuery and Cloud Storage)
Google Cloud provides specialized direct-export push subscriptions that stream messages directly into managed analytical sinks without requiring intermediate compute engines (such as Cloud Functions, Cloud Run, or Dataflow):
A. BigQuery Export Subscriptions
Functionality: Writes message payloads directly into a target BigQuery table using the BigQuery Storage Write API.
Configuration Options: Supports writing message metadata fields ( publish_time , message_id , attributes , subscription_name ) into reserved table columns. Can write the raw payload to a data column or map JSON fields directly to BigQuery table columns using topic schemas.
IAM Requirements: The Google-managed Pub/Sub Service Agent ( service-<PROJECT_NUMBER>@gcp-sa-pubsub.iam.gserviceaccount.com ) must be granted roles/bigquery.dataEditor on the destination table and roles/bigquery.metadataViewer on the dataset.
Cost Advantage: Incurs only Pub/Sub egress and BigQuery storage fees; zero intermediate VM or container compute costs are incurred.
B. Cloud Storage Export Subscriptions
Functionality: Batches messages and writes them directly into Cloud Storage buckets as files in Apache Avro or Text/JSON format.
Batching Parameters: Configurable by maximum time duration (e.g., flush every 5 minutes) and maximum file size (e.g., flush at 100 MB).
Path Formatting: Dynamic path variable prefixes allow structuring objects by date, hour, and minute (e.g., gs://my-bucket/logs/year=%Y/month=%m/day=%d/ ).
IAM Requirements: The Pub/Sub Service Agent requires roles/storage.objectCreator on the target bucket.
Feature / Dimension StreamingPull Synchronous Pull HTTPS Push BigQuery Direct Export
Initiator Subscriber opens gRPC stream Subscriber calls unary RPC Pub/Sub issues HTTPS POST Pub/Sub writes via API
Underlying Protocol Persistent gRPC (HTTP/2) Standard unary RPC Standard HTTPS POST Managed gRPC Storage API
Throughput & Latency Extreme throughput; sub-second Low throughput; high latency Moderate throughput; sub-second High throughput; batch stream
Auto-Scaling Model Consumer controls worker scaling Consumer schedules batch calls Consumer auto-scales on traffic Fully serverless auto-scaling
Authentication Standard Google Cloud IAM Standard Google Cloud IAM Google-signed OIDC JWT tokens Service Agent IAM roles
Firewall & Ingress Outbound internet / Private Access Outbound internet / Private Access Target must accept inbound HTTPS Internal Google Cloud network
Compute Overhead Worker VMs / Pods required Worker VMs / Scripts required Serverless container / Webhook Zero compute instances
Primary Exam Use Case Cloud Dataflow, Dataproc, GKE Batch cron extractors, scripts Cloud Run, Cloud Functions, SaaS Direct table ingest, ELT storage
Message Lifecycle, Retention Policies, and Replay (Seek)
Understanding how Pub/Sub tracks message lifecycle states prevents data loss and empowers automated pipeline disaster recovery. +-------------+
- | Published | ---> Persisted redundantly across availability zones in Colossus
- +-------------+
- |
- v
- +-------------+
- | Dispatched | ---> Delivered to subscriber; Ack Deadline timer begins
- +-------------+
- |
- +-------------------------------------+
- | |
- v (Worker completes task) v (Timeout / Worker crashes)
- +-------------+ +-------------+
- | ACK | | NACK / Exp |
- +-------------+ +-------------+
- | |
- | (retainAckedMessages = true) +---> Redelivered with backoff
- v
- +-----------------------------------+
- | Retained up to 31 Days for Replay |
- | (Seek to Timestamp or Snapshot) |
- +-----------------------------------+
- The Acknowledgement Deadline and Lease Management
When Pub/Sub delivers a message to a subscriber, it starts an acknowledgement deadline timer (configurable per subscription between 10 and 600 seconds, defaulting to 10 seconds). The subscriber must process the message and return an ACK before the deadline expires. If the subscriber crashes, hangs, or explicitly returns a negative acknowledgement ( NACK ), Pub/Sub releases the message lock and redelivers it.
Modern Google Cloud client libraries implement automatic lease management :
While subscriber worker threads remain actively executing a task, the client library issues background modifyAckDeadline requests to extend the deadline dynamically up to max_extension_duration (configurable up to several hours).
If a worker crashes or encounters an unhandled fatal error, the heartbeat halts immediately. The deadline lapses, and Pub/Sub re-dispatches the message to a surviving worker node.
Retention Policies
Unacknowledged Message Retention: Unacknowledged messages are retained in the subscription backlog for a configurable period between 10 minutes and 31 days (defaulting to 7 days). If no subscriber consumes them before the retention period expires, messages are permanently purged from the system.
Acknowledged Message Retention ( retainAckedMessages = true ): By default, Pub/Sub deletes a message immediately upon receiving an ACK . When retainAckedMessages is enabled on a subscription, Pub/Sub preserves acknowledged messages for the entire subscription retention window (up to 31 days). This retained buffer enables the Seek feature.
The Seek Feature: Replaying Messages and Snapshots
The Seek capability allows administrators and automated deployment scripts to rewind a subscription's delivery cursor:
Seek to Timestamp: Rewinds the subscription backlog to an exact historical UTC timestamp (e.g., "rewind cursor to 3 hours ago"). Any message published after that timestamp—whether previously acknowledged or unacknowledged—is marked unacknowledged and re-dispatched to active subscribers.
Seek to Snapshot: A Snapshot captures the exact unacknowledged state of a subscription at a specific instant. If an engineering team deploys a new version of a streaming Dataflow pipeline and discovers a transformation calculation bug 4 hours later, the team can deploy a code fix and execute a seek back to the pre-deployment snapshot. This cleanly reprocesses the exact event stream without dropping records or introducing data gaps.
Backlog Purging: In an incident where a faulty upstream producer floods a topic with millions of invalid test messages, an administrator can execute a seek to the current timestamp ( now ), immediately acknowledging and purging all backed-up messages to unblock consumers.
- Message State Description Transition Trigger
- Published / Stored Persisted across multiple zones; awaiting dispatch Publisher receives 200/gRPC OK
- In-Flight (Leased) Dispatched to worker; Ack deadline timer active Message dispatched via Pull or Push
- Acknowledged (ACK) Successfully processed by consumer Worker calls ack() or returns HTTP 200
- Negative Ack (NACK) Worker explicitly failed to process message Worker calls nack() or returns HTTP 5xx
- Expired Deadline Worker failed to respond before deadline elapsed Timer expires; rescheduled for redelivery
- Dead-Lettered Delivery attempts exceeded maxDeliveryAttempts Routed to dead-letter topic; ACKed on source
- Archived / Seekable Retained post-ACK via retainAckedMessages Preserved until retention window elapses
- Exactly-Once Delivery Semantics vs. At-Least-Once Default
By default, Cloud Pub/Sub guarantees at-least-once delivery . In distributed systems, temporary network splits, subscriber crashes, or acknowledgement timeouts can cause a message to be successfully processed by a worker while the corresponding ACK is lost in transit. Pub/Sub's safety guarantee ensures no message is ever lost, which means duplicate deliveries can occur.
Cloud Pub/Sub Exactly-Once Delivery
For pull subscriptions, Google Cloud offers a managed Exactly-Once Delivery feature. When enabled on a subscription:
Ack Deduplication: Pub/Sub ensures that once a message is acknowledged, any subsequent redelivery attempts are suppressed across the entire global routing layer.
In-Flight Deduplication: If a worker's acknowledgement deadline is being actively extended via StreamingPull lease management, Pub/Sub guarantees it will not deliver that same message to any other concurrent worker instance.
Success / Failure Ack Status: When a subscriber invokes acknowledge() , the gRPC response confirms whether the acknowledgment was accepted or whether the deadline had already elapsed, preventing silent double-writes.
Note
While Pub/Sub exactly-once delivery eliminates duplicate deliveries at the subscription boundary (within a 10-minute deduplication window), downstream sinks that require absolute transactional idempotency (such as inserting into BigQuery or relational databases) should still implement idempotent write patterns (e.g., using record primary keys, Spanner upserts, or BigQuery deterministic insert IDs).
Message Ordering and Ordering Keys
In standard topics, Pub/Sub maximizes global ingestion throughput by spreading messages across thousands of internal message broker shards. Consequently, messages are delivered to subscribers in arbitrary order, regardless of publish sequence.
However, transactional workloads—such as financial ledger debits and credits, inventory mutations, or user profile updates—demand strict sequential execution. Publisher sends events with Ordering Key = "acct_4502":
- [Event 1: Create Account] ---> [Event 2: Deposit $500] ---> [Event 3: Withdraw $200]
- |
- v
- Pub/Sub Ordering Guarantee
- |
- v
- Subscriber receives: Event 1 ==> Subscriber receives: Event 2 ==> Subscriber receives: Event 3
- (Event 2 is NEVER delivered until Event 1 is successfully ACKed)
- How Ordering Keys Function
Publishing with Keys: The publisher sets the orderingKey string attribute (e.g., user_id , device_serial_number , or account_id ) on each published message.
Subscription Ordering Enabled: The subscription must be configured with enableMessageOrdering = true .
Guaranteed Sequential Delivery: Pub/Sub ensures that messages bearing the same orderingKey are delivered to subscribers strictly in the order they were received and timestamped by the Pub/Sub service.
Per-Key Scalability: Ordering is enforced strictly per key , not across the entire topic. Two messages with different ordering keys (e.g., acct_4502 and acct_8911 ) are delivered concurrently in parallel across multiple worker threads. This provides linear horizontal scaling across millions of distinct entity IDs.
Head-of-Line Blocking Failure Semantics: If an error occurs while processing a message for an ordering key, or if the message is explicitly NACK'd, Pub/Sub halts delivery of all subsequent messages for that specific ordering key . Delivery remains paused until the blocking message is successfully processed and acknowledged, or until the message is routed to a dead-letter topic or purged via Seek. This behavior prevents state corruption and out-of-order execution in downstream databases.
Exam Traps and Antipatterns Summary
Antipattern / Trap Why It Fails Correct Exam Solution
Using Push subscriptions for massive multi-GB/s streaming HTTPS endpoint auto-scaling latency and HTTP per-message overhead cause timeouts and backpressure Deploy Pull subscriptions with StreamingPull (e.g., via Cloud Dataflow)
Polling Synchronous Pull in tight loops for stream ingest High network round-trip latency, thread blocking, and empty response billing waste compute Establish persistent bidirectional StreamingPull channels
Exposing public unauthenticated webhooks for Push subscriptions Allows unauthorized internet traffic to spoof events and overwhelm downstream services Configure OpenID Connect (OIDC) authentication on Push subscriptions with dedicated service accounts
Creating separate topics for every device/user to achieve ordering Exceeds project topic quotas, degrades performance, and creates immense administrative overhead Use a single topic with orderingKey per device/user and enable message ordering on subscriptions
Relying solely on client retry loops during pipeline crashes In-flight messages are lost if workers crash before re-queueing Enable retainAckedMessages and use Seek to rewind subscription cursors to timestamps or snapshots
Attempting to filter messages based on nested JSON payload fields Server-side Pub/Sub filters can only evaluate message attributes, not payload bytes Promote critical routing attributes into the message attributes map at publish time