Study guide
6.3 Windowing Strategies in Streaming Dataflow: Fixed, Sliding, and Session Windows
Google Cloud Certified Professional Data Engineer Exam
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Security, IAM, and Data Protection
1.1 Cloud IAM, Service Accounts, and Resource Hierarchy in GCP Data Systems 1.2 Data Encryption, Key Management (CMEK), and Cloud KMS 1.3 Sensitive Data Protection: Cloud DLP, De-identification, and Masking
2 Chapter 2: Reliability and Compliance Architecture
2.1 High Availability, Regional vs Multi-Regional Deployments, and Disaster Recovery 2.2 Data Residency, Sovereignty, and Regulatory Compliance (HIPAA, GDPR) 2.3 Modern Data Architecture Patterns: Data Mesh, Lakehouse, and Modern Data Stack
3 Chapter 3: Cross-Cloud and Federated Governance
3.1 BigLake Architecture: Unified Multicloud Storage and Fine-Grained Access Control 3.2 BigQuery Omni: Multicloud Analytics Across AWS S3, Azure ADLS, and Google Cloud 3.3 Analytics Hub and Federated Computational Governance
4 Chapter 4: Data Ingestion and Pipeline Planning
4.1 Data Pipeline Planning: Sources, Sinks, and Network Architecture (VPC, PSC, Interconnect) 4.2 Batch vs Streaming Processing Trade-Offs and Architecture Selection 4.3 Change Data Capture (CDC) with Datastream for Continuous Database Ingestion
5 Chapter 5: Streaming Ingestion with Cloud Pub/Sub
5.1 Cloud Pub/Sub Core Architecture: Topics, Subscriptions, and Message Lifecycle 5.2 Pub/Sub Message Ordering, Exactly-Once Delivery, and Deduplication 5.3 Pub/Sub BigQuery/Cloud Storage Direct Subscriptions and Dead-Letter Topics
6 Chapter 6: Pipeline Processing with Apache Beam and Cloud Dataflow
6.1 Apache Beam Programming Model: Pipelines, PCollections, and Core PTransforms 6.2 Cloud Dataflow Architecture: Autoscaling, Worker Allocation, and Dataflow Prime 6.3 Windowing Strategies in Streaming Dataflow: Fixed, Sliding, and Session Windows
7 Chapter 7: Advanced Dataflow: Robustness and Exactly-Once Semantics
7.1 Watermarks, Triggers, Accumulating vs Discarding Panes, and Late Data Handling 7.2 Side Inputs, Broadcast Joins, and Stateful Stream Processing 7.3 Pipeline Robustness: Dead-Letter Queues, Error Handling, and Pipeline Lifecycle Management
8 Chapter 8: Managed Data Processing: Cloud Dataproc and Cloud Data Fusion
8.1 Cloud Dataproc: Architecture, Ephemeral vs Persistent Clusters, and Spark Tuning 8.2 Dataproc Serverless and Autoscaling Policies for Batch Processing 8.3 Cloud Data Fusion: Visual ETL/ELT Pipelines, CDAP Plugins, and Data Lineage
9 Chapter 9: Storage Selection and Cloud Storage Architecture
9.1 GCP Storage Selection Decision Matrix: Relational, NoSQL, Object, and Analytical Stores 9.2 Cloud Storage Architecture: Storage Classes, Lifecycle Rules, and Object Versioning 9.3 Cloud Storage Security: Bucket Lock (WORM), Access Control, and Data Migration Tools
10 Chapter 10: Relational Cloud Databases: Cloud Spanner and Cloud SQL
10.1 Cloud Spanner Architecture: TrueTime, External Consistency, and Global Scalability 10.2 Cloud Spanner Schema Design: Primary Keys, Interleaved Tables, and Query Tuning 10.3 Cloud SQL: High Availability, Read Replicas, Maintenance, and Point-in-Time Recovery
11 Chapter 11: Scalable NoSQL Databases: Cloud Bigtable and Firestore
11.1 Cloud Bigtable Architecture: Storage Model, Tablet Splitting, and Cluster Management 11.2 Cloud Bigtable Row Key Design: Lexicographical Ordering, Salt Keys, and Anti-Patterns 11.3 Cloud Firestore: Document Model, Real-Time Sync, and ACID Collections
12 Chapter 12: Enterprise Data Warehousing with BigQuery
12.1 BigQuery Architecture: Capacitor Columnar Format, Jupiter Network, and Borg Slots 12.2 BigQuery Partitioning: Ingestion-Time, Date/Timestamp, and Integer Range Strategies 12.3 BigQuery Clustering: Cardinality, Multi-Column Ordering, and Performance Optimization 12.4 Data Warehouse Modeling: Normalization, Star Schemas, and Slowly Changing Dimensions
13 Chapter 13: Advanced Analytical Queries and Transformations in BigQuery
13.1 Advanced SQL in BigQuery: Window Analytic Functions, ARRAY/STRUCT, and PIVOT/UNPIVOT 13.2 Modern SQL Orchestration with Dataform: DAG Modeling, Testing, and Documentation 13.3 BI Engine In-Memory Acceleration and Enterprise Looker/LookML Semantic Modeling
14 Chapter 14: Machine Learning and Feature Engineering with BigQuery ML
14.1 BigQuery ML Architecture: In-Database Training, Model Topologies, and SQL Analytics 14.2 Feature Engineering: ML.TRANSFORM, Preprocessing Pipelines, and Vertex AI Feature Store 14.3 MLOps, Model Governance, Continuous Monitoring, and Deployment Lifecycles
15 Chapter 15: Operations, Orchestration, and Data Governance
15.1 Workflow Orchestration with Cloud Composer: Managed Airflow, DAG Design, and Dependencies 15.2 Unified Data Governance with Dataplex: Lakes, Zones, Data Quality, and Lineage 15.3 Pipeline Observability, FinOps, and Cost Optimization Across GCP Data Services
6.3 Windowing Strategies in Streaming Dataflow: Fixed, Sliding, and Session Windows
Key Takeaways Explain This Topic AI
Unbounded data streams must be processed using event time (the timestamp when the event occurred on the originating client) rather than processing time to produce deterministic, reproducible analytical results.
Fixed (tumbling) windows segment streams into contiguous, non-overlapping, uniform time intervals; each event belongs to exactly one window.
Sliding (hopping) windows create overlapping time intervals defined by duration and slide period; each element is duplicated across Duration / Slide windows, which can cause severe state explosion if the slide period is excessively small.
Session windows are dynamic, data-driven windows scoped strictly per key; they expand based on continuous bursts of activity and close after a specified gap duration of inactivity elapses.
The default Apache Beam trigger fires exactly once when the system watermark passes the end of the window; unhandled late-arriving events arriving after the watermark are permanently dropped unless allowed lateness is explicitly configured.
Last updated: September 2026
6.3 Windowing Strategies in Streaming Dataflow: Fixed, Sliding, and Session Windows
Exam Focus: The Google Cloud Professional Data Engineer exam rigorously tests your ability to select the correct windowing strategy based on business requirements. You must understand the difference between event time and processing time , avoid memory exhaustion caused by misconfigured sliding windows , understand the dynamic merging mechanics of session windows , and anticipate how the default trigger handles window boundaries.
In stream data processing, an unbounded collection represents an infinite, continuously arriving sequence of records. Because an unbounded stream has no end, global operations—such as calculating a total sum, finding the minimum value, or counting records—cannot be performed across the entire stream at once without subdividing the data into finite, discrete temporal chunks. This temporal partitioning process is known as Windowing .
1. Time Semantics: Event Time vs. Processing Time
To understand windowing, data engineers must distinguish between two fundamental temporal domains: +─────────────────────────────────────────────────────────────────────────────+
- | TIME DOMAINS IN STREAMING |
- +─────────────────────────────────────────────────────────────────────────────+
- | |
- | EVENT TIME: 12:01:05 PM PROCESSING TIME: 12:04:30 PM |
- | (Occurred on mobile device) (Arrived at Dataflow worker) |
- | │ ▲ |
- | │ │ |
- | └────────────[ Network Delay / Skew ]──────────┘ |
- | (3m 25s Time Lag) |
- +─────────────────────────────────────────────────────────────────────────────+
Event Time: The timestamp assigned to an event when it physically occurred on the originating client device, sensor, or operational database. This timestamp is embedded directly in the payload or message metadata.
Processing Time: The wall-clock time of the compute infrastructure (the Dataflow worker instance) at the precise instant the element is processed by an execution transform.
Why Event Time is Mandatory for Accurate Analytics
In real-world networks, events never arrive instantaneously or in perfect chronological order. Network latency, intermittent cellular connectivity, mobile device offline caching, and regional network partitions introduce unpredictable time skew between event time and processing time.
If an analytics pipeline calculates 5-minute sales metrics using processing time , a mobile transaction executed at 12:01 PM that is delayed by intermittent connectivity until 12:15 PM would be incorrectly attributed to the 12:15 PM window. Processing time yields non-deterministic results that change whenever a pipeline is paused, re-run, or backfilled. Event time provides deterministic, reproducible analytics regardless of network delays.
Timestamp Assignment in Beam
Sources such as PubsubIO.readMessagesWithAttributes() automatically extract the Pub/Sub message publish timestamp as the Beam event timestamp.
If the event timestamp is stored in a custom payload attribute (e.g., event_timestamp ), developers use the WithTimestamps transform to assign the custom event timestamp to each record before windowing. // Assigning custom event time from a JSON payload
- PCollection<TelemetryEvent> timestampedEvents = rawEvents.apply(
- "AssignEventTime",
- WithTimestamps.of((TelemetryEvent event) -> Instant.ofEpochMilli(event.getDeviceTimestamp()))
- );
- 2. Windowing Fundamentals on Unbounded Collections
In Apache Beam, every element in a PCollection is associated with an implicit timestamp and assigned to one or more windows.
The GlobalWindow (Default)
By default, all elements in any newly created PCollection belong to a single, all-encompassing GlobalWindow spanning [ − ∞ , + ∞ ) [-\infty, +\infty) [ − ∞ , + ∞ ) .
In a bounded (batch) pipeline, performing an aggregation on the GlobalWindow succeeds because the runner processes all data to completion and emits the final result when the batch source reaches end-of-file.
In an unbounded (streaming) pipeline, performing a grouping or aggregation (such as GroupByKey or CombinePerKey ) on the default GlobalWindow without configuring custom triggers will never emit any output . Because the stream never terminates, the global window never closes.
To perform aggregations on an unbounded stream, developers must apply a non-global windowing strategy: Fixed Windows , Sliding Windows , or Session Windows .
3. Fixed (Tumbling) Windows
Fixed windows (also referred to as tumbling windows) segment the stream into continuous, non-overlapping, uniform time intervals. Stream Timeline (Event Time) ─────────────────────────────────────────────────>
- [ Window 1: 12:00 - 12:05 ) [ Window 2: 12:05 - 12:10 ) [ Window 3: 12:10 - 12:15 )
- Event A (12:02) • Event C (12:06) • Event E (12:11)
- Event B (12:04) • Event D (12:09) • Event F (12:14)
- Architectural Characteristics
Non-Overlapping: Windows are contiguous. An event belongs to exactly one window based on its event timestamp t t t .
Formula: A window of duration D D D starting at boundary W 0 W_0 W 0 contains all events with timestamp t t t satisfying: W start ≤ t < W start + D W_{\text{start}} \le t < W_{\text{start}} + D W start ≤ t < W start + D .
State Lifecycle: When the watermark advances past the window end, the window fires its aggregation result and its state is reclaimed from memory.
Typical Use Cases: Hourly billing rollups, 5-minute average website latency dashboards, daily summaries, regular periodic metrics. // Implementing a 5-minute Fixed Window in Apache Beam
- PCollection<KV<String, Long>> fiveMinuteCounts = inputEvents
- .apply("FixedWindows", Window.into(FixedWindows.of(Duration.standardMinutes(5))))
- .apply("CountPerKey", Count.perKey());
- 4. Sliding (Hopping) Windows
Sliding windows (also known as hopping windows) represent overlapping, fixed-duration time intervals defined by two parameters: Window Duration and Slide Period . Duration = 10 Minutes, Slide Period = 5 Minutes
- Window 1: [ 12:00 ─────────────────────> 12:10 )
- Window 2: [ 12:05 ─────────────────────> 12:15 )
- Window 3: [ 12:10 ─────────────────────> 12:20 )
- ▲
- Event X (12:07)
- (Belongs to BOTH Window 1 AND Window 2!)
- Architectural Characteristics
Overlapping Intervals: Because the slide period is smaller than the window duration ( S < D S < D S < D ), consecutive windows overlap in time.
Multi-Window Assignment: Each incoming element is assigned to multiple concurrent windows simultaneously.
Duplication Multiplier: The exact number of overlapping windows to which each element is assigned is given by: N = Window Duration Slide Period N = \frac{\text{Window Duration}}{\text{Slide Period}} N = Slide Period Window Duration
The Memory Explosion Anti-Pattern
A frequent production mistake on streaming Dataflow pipelines is setting an excessively small slide period relative to the window duration.
The Trap: An engineer specifies a 1-hour window sliding every 1 second ( D = 3600 s , S = 1 s D = 3600\text{s}, S = 1\text{s} D = 3600 s , S = 1 s ). In this scenario, N = 3600 / 1 = 3 , 600 N = 3600 / 1 = 3,600 N = 3600/1 = 3 , 600 .
Impact: Every single incoming event is duplicated across 3,600 separate in-memory window buffers. If the stream processes 10,000 events/sec, the pipeline attempts to write and aggregate 36,000,000 window states per second, rapidly crashing worker VMs or Streaming Engine backend storage with catastrophic Out-of-Memory exceptions.
Best Practice: Keep the ratio D / S D / S D / S modest (typically between 2 2 2 and 20 20 20 , such as a 10-minute window sliding every 1 minute).
Typical Use Cases
Moving averages (e.g., 15-minute moving average temperature updated every 1 minute).
Rolling anomaly detection (e.g., alert if error rate over the last 30 minutes exceeds 5%, evaluated every 2 minutes).
Trend smoothing in financial trading tickers. // Implementing a 10-minute Sliding Window updated every 1 minute
- PCollection<KV<String, Double>> movingAverage = sensorReadings
- .apply("SlidingWindows", Window.into(
- SlidingWindows.of(Duration.standardMinutes(10))
- .every(Duration.standardMinutes(1))
- ))
- .apply("AveragePerSensor", Mean.perKey());
- 5. Session Windows
Session windows are dynamic, data-driven windows defined by periods of continuous activity separated by a specified Gap Duration of inactivity. User 1: [ Event ]──[ Event ]──[ Event ] ──(Inactivity > 30m)──> [ Window Closes: Length 45m ]
- User 2: [ Event ] ────────────────────────(Inactivity > 30m)──> [ Window Closes: Length 0m ]
- Architectural Characteristics
Strictly Scoped Per Key: Session windows do not apply globally across a dataset. They are scoped strictly per key (e.g., user_id or session_token ). Different keys have completely independent window start times, durations, and end boundaries.
Data-Driven Boundaries: Unlike fixed or sliding windows, session windows have no fixed calendar start or end points. They expand and contract based on incoming event intervals.
Dynamic Window Merging Mechanics:
When an event arrives at timestamp t t t , Beam creates an initial ephemeral window [ t , t + gap ) [t, t + \text{gap}) [ t , t + gap ) .
When a subsequent event arrives for the same key at timestamp t 2 t_2 t 2 (where t 2 < t + gap t_2 < t + \text{gap} t 2 < t + gap ), its window [ t 2 , t 2 + gap ) [t_2, t_2 + \text{gap}) [ t 2 , t 2 + gap ) overlaps with the existing window.
Apache Beam's windowing engine dynamically merges the two overlapping windows into a single expanded window spanning [ t , t 2 + gap ) [t, t_2 + \text{gap}) [ t , t 2 + gap ) .
Merging continues dynamically until a period of inactivity greater than or equal to the gap duration elapses, at which point the session window finalizes.
Typical Use Cases
Tracking e-commerce user browsing sessions (e.g., group all user clicks until 30 minutes of inactivity, then calculate total session spend or abandoned carts).
Mobile gaming play sessions.
IoT connected vehicle trips (from engine start until parked for > 15 minutes). // Implementing Session Windows with a 30-minute Gap Duration
- PCollection<KV<String, Long>> sessionCounts = userClicks
- .apply("SessionWindows", Window.into(Sessions.withGapDuration(Duration.standardMinutes(30))))
- .apply("CountClicksPerSession", Count.perKey());
- 6. Single-Key vs. Global Aggregations & Trigger Interactions
Understanding how windowing interacts with keys and triggers is vital for exam scenario questions.
Keyed vs. Non-Keyed Window Aggregations
Keyed PCollections ( PCollection<KV<K, V>> ): When windowing is applied to a keyed PCollection, window state and aggregations are managed independently per key. One key can fire its window while another key continues accumulating state. This enables horizontal parallelization across all cluster workers.
Non-Keyed PCollections ( PCollection<V> ): Applying a global aggregation (such as Count.globally() ) across a windowed stream forces all elements for that window across the entire stream onto a single worker node , creating an extreme performance bottleneck.
Default Trigger Behavior and Window Firing
By default, Apache Beam attaches the Default Trigger to any windowed PCollection: Default Trigger = Repeatedly ( AfterWatermark . pastEndOfWindow ( ) ) \text{Default Trigger} = \text{Repeatedly}\left(\text{AfterWatermark}.\text{pastEndOfWindow}()\right) Default Trigger = Repeatedly ( AfterWatermark . pastEndOfWindow ( ) )
Trigger Firing: The default trigger fires exactly once when the system watermark passes the end of the window ( W end W_{\text{end}} W end ). At this instant, the aggregated pane is emitted downstream to sinks.
Late Data Handling under Default Trigger: If an event arrives with an event timestamp belonging to an already-closed window (because its event timestamp is earlier than the current watermark), the default trigger permanently drops the late event .
Allowed Lateness: To retain and process late-arriving events, engineers must explicitly configure .withAllowedLateness(Duration) and specify accumulating or discarding pane accumulation modes.
7. Comparative Windowing Matrix & Production Scenarios
Window Type Boundary Type Overlapping? Elements Belong To Memory Footprint Primary Business Use Case
Fixed (Tumbling) Static (Clock-aligned) No Exactly 1 window Moderate Hourly billing, 5-minute dashboards, regular periodic rollups.
Sliding (Hopping) Static (Clock-aligned) Yes Multiple ( D / S D / S D / S windows) High to Extreme Moving averages, rolling anomaly detection, trend smoothing.
Session Dynamic (Data-driven) Merged dynamically Exactly 1 merged window per key Moderate User session analytics, gaming sessions, IoT trip logging.
Global Static ( − ∞ , + ∞ -\infty, +\infty − ∞ , + ∞ ) N/A Exactly 1 global window Low (Batch) / High (Stream) Batch aggregations; streaming requires custom triggers.
Realistic Exam Scenarios & Architecture Pitfalls
Scenario / Problem Anti-Pattern Correct Google Cloud Architecture
E-Commerce Inactivity Tracking ; An online retailer needs to analyze customer browsing sessions. A session is defined as continuous user clicks where any inactivity period exceeding 20 minutes terminates the session. Using 20-minute Fixed Windows to group user events. Apply Session Windows with a 20-minute gap duration ( Sessions.withGapDuration(Duration.standardMinutes(20)) ) on a keyed PCollection ( KV<UserId, ClickEvent> ). Windows dynamically merge continuous activity per user.
Rolling Metric CPU / RAM Exhaustion ; A streaming pipeline computes a 30-minute moving average of network packet volume, updated every 2 seconds. The pipeline crashes with memory exhaustion across all workers. Configuring sliding windows with a 30-minute duration and a 2-second slide ( D / S = 900 D/S = 900 D / S = 900 overlapping windows per event). Increase the slide period to a reasonable business frequency (e.g., 30-second or 1-minute slide, reducing N N N from 900 to 30), or use an external rolling time-series database (e.g., Cloud Bigtable) to compute micro-aggregates.
Processing Time Skew in Telemetry ; An IoT vehicle monitoring pipeline uses processing time windowing to aggregate engine temperature readings. Vehicles traveling through tunnels upload 2 hours of buffered data upon re-emerging, creating false spike alerts. Relying on processing-time windows or default system ingestion timestamps. Assign event timestamps using WithTimestamps based on the sensor's recorded device timestamp. Apply Fixed Windows in event time so tunnel readings correctly fall into their historical 5-minute windows.
Default Trigger Late Data Loss ; Fleet GPS telemetry buffered during mountain tunnel transits arrives 25 minutes late and disappears from fixed-window aggregations under default settings. Assuming late records are automatically queued and merged without pipeline configuration. Configure .withAllowedLateness(Duration.standardHours(1)) combined with .triggering(AfterWatermark.pastEndOfWindow().withLateFirings(...)) to retain window state in Streaming Engine and emit late corrective panes.