Study guide
3.6 Ingesting Structured and Unstructured Data into Training Pipelines
Google Cloud Professional Machine Learning Engineer Exam
Track your progress as you read
Finished sections get a check mark.
1 Introduction & Exam Overview
0.1 Exam Overview, Blueprint & Certification Logistics 0.2 Study Strategy, Scenario Mechanics & 8-Week Roadmap
2 Section 1: Architecting Low-Code AI Solutions (~13%)
1.1 BigQuery ML: Model Types, SQL Syntax and In-Database Evaluation 1.2 Feature Engineering and Feature Selection in BigQuery ML 1.3 Generating Predictions with BigQuery ML 1.4 AutoML on Agent Platform: Tabular, Image, Text and Video Training 1.5 Fine-Tuning Gemini Models from BigQuery 1.6 Model Garden: Evaluating and Selecting the Right Model 1.7 Industry-Specific AI APIs: Document AI, Vision, Translate and Speech 1.8 Generative Media and Models as a Service: Gemini, Imagen and Veo 1.9 Optimizing Gemini Applications for Cost, Latency and Availability
3 Section 2: Collaborating Within and Across Teams to Manage Data and Models (~16%)
2.1 Organizing and Exploring Tabular, Text, Image and Video Data 2.2 Choosing the Preprocessing Tool: BigQuery, Dataflow, Spark and In-Memory Python 2.3 Creating and Consolidating Features in Agent Platform Feature Store 2.4 Data Privacy, PII Handling and Sensitive Information 2.5 Notebook Environments: Collaboration and Security Best Practices 2.6 Prototyping in Workbench and Colab Enterprise with PyTorch, scikit-learn and JAX 2.7 Prototyping with Model Garden Foundational and Open Models in Notebooks 2.8 Choosing the Development and Experimentation Environment 2.9 Evaluating Generative Solutions: LLM-as-a-Judge and RAG Evaluation 2.10 Experiment Tracking, Artifact Lineage and Model Registry Governance
4 Section 3: Scaling Prototypes into ML Models (~21%)
3.1 Choosing the Model Type: ARIMA, DNN, LLM and Classical Baselines 3.2 Choosing the Product: AutoML, BigQuery ML, Custom Training and Pipelines 3.3 Choosing the Deployment Strategy at Design Time 3.4 Modeling Techniques for Interpretability Requirements 3.5 Organizing Training Data on Cloud Storage and BigQuery 3.6 Ingesting Structured and Unstructured Data into Training Pipelines 3.7 Training SDKs and Services: Custom Training, Kubeflow on GKE, AutoML and Tabular Workflows 3.8 Troubleshooting ML Model Training Failures 3.9 Hyperparameter Tuning, Bayesian Optimization and Vizier 3.10 Model Evaluation Metrics, Transfer Learning and Fine-Tuning 3.11 Evaluating Compute and Accelerator Options: CPU, GPU and TPU 3.12 Distributed Training Strategies, Hardware Acceleration and Performance Optimization
5 Section 4: Serving and Scaling Models (~20%)
4.1 Batch and Online Inference: Agent Platform, Model Garden, Cloud Run and GKE 4.2 Rollout Strategies: Canary, Traffic Splitting, Shadow and Blue-Green 4.3 Tuning Models for Production: Vector Search and Model Compression 4.4 Deploying Models to Public and Private Endpoints 4.5 Choosing Serving Hardware: CPU, GPU, TPU and Edge 4.6 Scaling the Serving Backend for Throughput
6 Section 5: Automating and Orchestrating ML Pipelines (~18%)
5.1 Validating Data and Models Inside Pipelines 5.2 Agent Platform Pipelines: Kubeflow Pipelines (KFP) and TFX Architecture 5.3 Reusable Pipeline Components, Artifacts and ML Metadata Lineage 5.4 Ray on Agent Platform for Distributed Python Workloads 5.5 MLOps CI/CD Automation, Retraining Triggers and Skew Prevention
7 Section 6: Monitoring AI Solutions (~13%)
6.1 Securing AI Systems: Exfiltration, Malicious Prompting and Model Armor 6.2 Responsible AI, Explainable AI, Fairness Metrics and Model Governance 6.3 Model Monitoring: Feature Skew and Drift Detection 6.4 Operational Troubleshooting, Performance Profiling and Cost Optimization 6.5 Monitoring, Testing and Evaluating Generative AI in Production
3.6 Ingesting Structured and Unstructured Data into Training Pipelines
Key Takeaways Explain This Topic AI
Batch ingestion into BigQuery or Cloud Storage suits bounded historical loads; Pub/Sub with Dataflow suits unbounded streams that must be processed continuously.
Dataflow runs the same Apache Beam pipeline in batch and streaming mode, which is what makes one transformation definition serve both training and serving paths.
The BigQuery Storage Read API streams table data to training workers in parallel Arrow batches without materializing an export.
Ingestion must be idempotent and deduplicated, because at-least-once delivery means duplicate records will arrive and duplicates bias training.
Late-arriving and out-of-order events require windowing with watermarks; ignoring them produces silently incomplete aggregates.
Last updated: September 2026
3.6 Ingesting Structured and Unstructured Data into Training Pipelines
Blueprint reference: Section 3.2, "Ingesting structured and unstructured data from various sources into training pipelines."
Ingestion is where a large share of production ML defects originate, because it is the point at which data changes shape, arrives twice, or arrives late. The exam tests both the service selection and the correctness concerns.
Choosing the Ingestion Path
- Source and shape Path Notes
- Files in Cloud Storage, bounded BigQuery load job or direct read Cheapest for one-off historical loads
Operational database, periodic Datastream into BigQuery, or a scheduled extract Change data capture keeps a replica current
Unbounded event stream Pub/Sub → Dataflow → BigQuery / Cloud Storage The canonical streaming ingestion path
Continuous rows into BigQuery BigQuery Storage Write API High-throughput streaming ingestion with exactly-once semantics
- Existing Spark or Hadoop jobs Dataproc Lift-and-shift; keeps existing code
- Complex transformation, batch or stream Dataflow (Apache Beam) One pipeline definition for both modes
Orchestration of heterogeneous steps Managed Service for Apache Airflow Scheduling and dependencies across systems
Reading BigQuery into a training loop BigQuery Storage Read API Parallel Arrow streams; no export step
Unstructured files for training Cloud Storage , catalogued by an object table Bytes in object storage, metadata in SQL
Why Dataflow Keeps Appearing
The reason Dataflow dominates this area on the exam is a specific property: the same Apache Beam pipeline runs in batch and in streaming mode . That matters for ML because the transformation applied to historical training data and the transformation applied to live serving data can be the same code , which eliminates a whole class of training-serving skew.
Beam gives you:
Unified batch and streaming semantics , including windowing.
Watermarks and triggers for handling late and out-of-order data explicitly.
Autoscaling and dynamic work rebalancing so a straggling shard does not stall the job.
Exactly-once processing semantics within the pipeline.
When a scenario says "the same preprocessing must apply to the nightly training extract and to the live stream," Dataflow is the intended answer.
Reading BigQuery into Training Efficiently
Training loops that pull from BigQuery through the standard query API are slow and can hit result-size limits. The BigQuery Storage Read API streams table data directly to readers as Arrow record batches over multiple parallel streams, which maps naturally onto multiple training workers each taking a stream. # Each worker reads its own stream; no export to Cloud Storage required
- from google.cloud import bigquery_storage
- client = bigquery_storage.BigQueryReadClient()
- session = client.create_read_session(
- parent=f"projects/{project}",
- read_session={"table": table_path,
- "data_format": bigquery_storage.DataFormat.ARROW},
- max_stream_count=num_workers,
- )
The alternative — export to Parquet or TFRecord in Cloud Storage, then read the files — is preferable when the same extract will be read many times across many runs, because the export cost is paid once and sequential file reads are extremely fast.
Correctness Concerns
Idempotency and deduplication. Pub/Sub delivers at least once . A publisher retry, a subscriber redelivery, or a pipeline restart will produce duplicate messages. Duplicates in training data over-weight the duplicated examples and inflate any evaluation that shares them. The standard defences:
A stable message-level identifier carried in an attribute, deduplicated in a window.
Idempotent writes keyed on a natural business key, so a repeat write overwrites rather than appends.
The BigQuery Storage Write API with stream offsets for exactly-once ingestion.
Late and out-of-order events. In a streaming pipeline, an event's processing time is not its event time . Aggregating on arrival produces aggregates that quietly omit late data. Beam's watermarks track how complete the event-time view is, windows define the aggregation boundaries, and triggers and allowed lateness define what happens when data arrives after the window closes. For ML, late data matters because a feature computed as "transactions in the last hour" must be reproducible offline; if the streaming version silently dropped late events and the batch version included them, the two disagree.
Schema evolution. Sources add and rename fields. Defences:
Use formats that carry a schema: Avro (designed for evolution), Parquet, or Protocol Buffers.
Validate incoming data against an expected schema at the pipeline boundary and route violations to a dead-letter destination rather than failing the job or, worse, silently coercing.
Version the schema and record which version each training run consumed.
Dead-letter handling. Any record that cannot be parsed or validated should be written to a dead-letter table or bucket with the error and the raw payload. A pipeline that drops bad records silently is indistinguishable from one that is working, until a model degrades for reasons nobody can trace.
PII at ingestion. Inspect and de-identify at the boundary rather than after the data has landed in several places; this is where Sensitive Data Protection belongs in the flow.
Unstructured Ingestion
For images, audio, video, and documents, the pattern is consistent: land the bytes in Cloud Storage, land the metadata in BigQuery . Ingestion then means writing the object and inserting a metadata row transactionally enough that the catalogue does not diverge from the bucket. Object tables give the SQL view over what actually exists, which is the reconciliation tool when they do diverge.
Exam Traps
Ignoring at-least-once delivery. Duplicates will occur; dedupe explicitly.
Aggregating on processing time when event time is what matters.
Failing the whole job on one bad record instead of dead-lettering it.
Separate batch and streaming transformation code. Guarantees skew; use one Beam pipeline.
Querying BigQuery row by row from a training loop instead of using the Storage Read API or an export.