Study guide
11.4 Unstructured Data, Embeddings, and Retrieval-Augmented Generation
Google Cloud Professional Data Engineer Exam
Track your progress as you read
Finished sections get a check mark.
1 Introduction & Exam Overview
0.1 About the Google Cloud Professional Data Engineer Exam 0.2 Study Strategy & Cloud Data Engineering Methodology
2 Chapter 1: Designing for Security and Compliance
1.1 IAM Roles, Service Accounts, and Least Privilege for Data 1.2 Data Encryption, Key Management, and Privacy with Cloud DLP 1.3 Compliance, Data Sovereignty, and Geographic Placement 1.4 Multi-Environment Architecture: Development, Staging, and Production
3 Chapter 2: Designing for Reliability and Resilience
2.1 High Availability, Disaster Recovery, and Failover Patterns 2.2 Fault Tolerance and ACID Guarantees in Distributed Systems 2.3 Data Validation, Cleansing, and Quality Assurance with Dataform 2.4 AI-Assisted Data Preparation and LLM Query Generation
4 Chapter 3: Architecture Patterns and Flexibility
3.1 Modern Data Architectures: Data Lake, Warehouse, Lakehouse, and Mesh 3.2 Data Migration Strategies and Hybrid/Multi-Cloud Integration 3.3 Database Migration Service and Heterogeneous Database Migrations
5 Chapter 4: Ingestion Architectures: Pub/Sub and Streaming Sources
4.1 Cloud Pub/Sub Core Architecture and Ingestion 4.2 Pub/Sub Advanced Patterns: Dead-Lettering, Ordering, and Schemas 4.3 Change Data Capture (CDC) with Datastream and Batch Ingestion
6 Chapter 5: Stream and Batch Processing with Apache Beam and Cloud Dataflow
5.1 Apache Beam Core Concepts and Pipeline Design 5.2 Advanced Windowing, Watermarks, and Triggers 5.3 Dataflow Pipeline Optimization, Scaling, and Templates
7 Chapter 6: Data Transformation with Dataproc, Data Fusion, and Workflow Orchestration
6.1 Apache Spark and Hadoop with Cloud Dataproc 6.2 Visual Data Pipelines with Cloud Data Fusion 6.3 Workflow Orchestration with Cloud Composer 6.4 AI Data Enrichment inside Data Pipelines
8 Chapter 7: Storage Service Selection and Cloud Storage Architecture
7.1 Google Cloud Storage Service Selection Decision Matrix 7.2 Cloud Storage Architecture, Classes, and Lifecycle Management
9 Chapter 8: Deep Dive into Google BigQuery Storage
8.1 BigQuery Storage Architecture and Data Organization 8.2 Partitioning and Clustering for Performance and Cost 8.3 BigLake and Hybrid Querying across Clouds 8.4 Data Warehouse Modeling: Star Schemas, Denormalization, and Slowly Changing Dimensions
10 Chapter 9: Operational and NoSQL Databases: Bigtable and Cloud Spanner
9.1 Cloud Bigtable Architecture and Row Key Design 9.2 Cloud Bigtable Schema Management and Performance Tuning 9.3 Cloud Spanner: Globally Scalable Relational Database
11 Chapter 10: BigQuery Query Performance and Optimization
10.1 BigQuery SQL Tuning and Performance Best Practices 10.2 BigQuery BI Engine and Materialized Views 10.3 Data Sharing and Analytics Hub 10.4 Preparing Data for Visualization and BI Tool Connectivity
12 Chapter 11: Machine Learning Readiness and Data Governance
11.1 Machine Learning with BigQuery ML 11.2 Feature Engineering and Data Preparation for AI/ML 11.3 Data Governance, Cataloging, and Discovery with Dataplex 11.4 Unstructured Data, Embeddings, and Retrieval-Augmented Generation
13 Chapter 12: Site Reliability Engineering and Monitoring for Data Systems
12.1 SRE Principles for Data: SLIs, SLOs, and SLAs 12.2 Observability: Cloud Monitoring, Cloud Logging, and Alerting 12.3 Troubleshooting Data Pipelines and Workload Failures
14 Chapter 13: Workload Automation, CI/CD, and Cost Optimization
13.1 CI/CD and Version Control for Data Pipelines 13.2 BigQuery Cost Management and Capacity Planning 13.3 Holistic Cost Optimization across GCP Data Services 13.4 Query Workload Management: Interactive vs. Batch Jobs, Queues, and Quotas
11.4 Unstructured Data, Embeddings, and Retrieval-Augmented Generation
Key Takeaways Explain This Topic AI
Preparing unstructured data for embeddings and retrieval-augmented generation is an explicit v4.2 blueprint bullet, and it is tested as a data pipeline problem: parse, chunk, embed, index, retrieve, and keep fresh.
Unstructured files are exposed to BigQuery through object tables and parsed with ML.PROCESS_DOCUMENT, ML.TRANSCRIBE, or ML.ANNOTATE_IMAGE before ML.GENERATE_EMBEDDING produces the vector column.
BigQuery vector indexes come in two types: IVF, which k-means clusters and partitions the vectors and suits small query batches, and TreeAH, built on ScaNN with product quantization for large query batches; distance types are EUCLIDEAN (default), COSINE, and DOT_PRODUCT.
A table smaller than 10 MB will not have its vector index populated and searches fall back to brute force, newly inserted rows are brute-forced rather than missed, and index stored columns are ignored when the table has a row-level access policy or the column has a policy tag.
Pre-filtering narrows candidates before the nearest-neighbor search so top_k is drawn from the filtered population, while post-filtering can return far fewer than top_k rows; VECTOR_SEARCH and AI.SEARCH are not accelerated by BI Engine.
Last updated: September 2026
11.4 Unstructured Data, Embeddings, and Retrieval-Augmented Generation
Sub-section 4.2 of the v4.2 exam guide has exactly two bullets. The first is feature engineering and model training with BigQuery ML, covered earlier in this chapter. The second is new and catches most candidates unprepared: "Preparing unstructured data for embeddings and retrieval-augmented generation (RAG)."
Read the verb. The blueprint does not ask you to build a chatbot; it asks you to prepare the data that a retrieval system depends on. That is a data engineering job with a familiar shape — parse, chunk, transform, index, keep fresh, govern — and the exam tests it as such.
- The Five Stages of a RAG Data Pipeline Cloud Storage objects
- |
- (1) Parse -> object table + ML.PROCESS_DOCUMENT / ML.TRANSCRIBE / ML.ANNOTATE_IMAGE
- |
- (2) Chunk -> split into retrievable passages + carry source metadata
- |
- (3) Embed -> ML.GENERATE_EMBEDDING -> ARRAY<FLOAT64> column
- |
- (4) Index -> CREATE VECTOR INDEX (IVF or TreeAH)
- |
- (5) Retrieve -> VECTOR_SEARCH / AI.SEARCH -> ground ML.GENERATE_TEXT
- with the retrieved passages
- Stage 1: Parse
Unstructured source files live in Cloud Storage. BigQuery reaches them through an object table — a table of references to the underlying objects — and the extraction functions read from there:
ML.PROCESS_DOCUMENT extracts structured fields and text from PDFs and scanned documents through Document AI.
ML.TRANSCRIBE converts audio to text through Speech-to-Text.
ML.ANNOTATE_IMAGE returns Cloud Vision labels, objects, and detections.
Google documents this whole flow as achievable inside BigQuery: parse documents, run vector search over the content, and generate summarized answers to natural-language questions with Gemini models.
Stage 2: Chunk
A document is almost never the right retrieval unit. A 90-page policy manual embedded as one vector produces a single, blurry point in embedding space that matches everything and answers nothing. Chunking splits the text into passages small enough to be semantically specific and small enough to fit in the generation model's context alongside several siblings.
Three engineering decisions matter:
Decision Trade-Off
Chunk size Small chunks retrieve precisely but fragment context; large chunks preserve context but dilute the embedding and crowd the context window
Overlap Overlapping adjacent chunks prevents an answer being split across a boundary, at the cost of duplicate storage and near-duplicate retrieval hits
Chunk metadata Carrying the source URI, page or section, document date, and tenant or department on every chunk is what makes citation, filtering, and access control possible later
That last row is the one with real exam consequences. Metadata columns stored alongside the embedding are what you filter on in stage 5 — without them you cannot restrict a search to one tenant, one language, or documents newer than a given date.
Stage 3: Embed
ML.GENERATE_EMBEDDING produces the vector column. It operates on structured text in ordinary columns and, through ObjectRef values, on text, image, and video inputs. The output is an array of floats stored alongside the chunk.
The data-engineering obligations here are unglamorous and heavily testable:
Embeddings and the model that produced them are coupled. Different embedding models produce vectors of different dimensionality and different geometry. Switching models means re-embedding the entire corpus , not embedding new rows with the new model and leaving old rows alone — a mixed-model table returns meaningless distances.
Freshness is a pipeline problem. When a source document is revised, its chunks must be re-chunked and re-embedded. An incremental pipeline keyed on object generation or update time is the standard pattern.
De-duplicate before embedding. Embedding the same boilerplate footer 40,000 times wastes both the embedding call and the retrieval slots it will later occupy.
Stage 4: Index
Without an index, VECTOR_SEARCH performs a brute-force scan and returns exact results. With a vector index it uses approximate nearest neighbor search, which is far faster at the cost of some recall.
BigQuery offers two index types:
Index Type Mechanism Preferred For
IVF Inverted file index; k-means clusters the vectors, then partitions the data by cluster so the search reads only relevant partitions Small query batches
TreeAH Built on Google's ScaNN algorithm; shards the base table, trains a clustering model sized by leaf_node_embedding_count , and compresses vectors with product quantization Large query batches
Supported distance types are EUCLIDEAN (the default), COSINE , and DOT_PRODUCT . Index training always uses Euclidean distance internally, but the distance used at search time can differ, and a distance_type passed to VECTOR_SEARCH overrides the index's default.
Three index behaviors that make excellent exam questions:
A table smaller than 10 MB will not have its vector index populated. If an indexed table shrinks below 10 MB, the index is temporarily disabled and searches report BASE_TABLE_TOO_SMALL . The search still works — it falls back to brute force.
Newly added rows are never missed. There is a delay between inserting rows and their appearance in the index, but VECTOR_SEARCH and AI.SEARCH search the index for indexed records and brute-force the not-yet-indexed ones.
Stored columns are ignored when security is applied. The STORING clause caches metadata columns in the index for efficient pre-filtering, but stored columns are not used if the table has a row-level access policy or if the column carries a policy tag.
Stage 5: Retrieve and Ground
VECTOR_SEARCH returns the nearest chunks; AI.SEARCH provides semantic or hybrid search against a string, which is convenient when autonomous embedding generation is enabled on the table. The retrieved passages are then passed as context into ML.GENERATE_TEXT or AI.GENERATE so the model answers from your corpus rather than from its training data.
Pre-filter versus post-filter is the retrieval-quality decision: a pre-filter narrows the candidate set before the nearest-neighbor search, so you still get top_k results from within the filtered population; a post-filter applies the condition to results the search already returned, which can leave you with far fewer than top_k rows — or none. When a scenario complains that a tenant-scoped search returns too few results, the answer is to pre-filter.
One cost note worth memorizing: VECTOR_SEARCH and AI.SEARCH queries are not accelerated by BI Engine , so an in-memory reservation will not rescue a slow semantic search dashboard.
BigQuery Vector Search vs. Vertex AI Vector Search
- Requirement Choose
- Corpus already in BigQuery; analytical or batch retrieval; SQL-only team BigQuery vector search
- Retrieval joined to warehouse facts and dimensions in the same query BigQuery vector search
- Online serving at high QPS with single-digit millisecond latency for an application Vertex AI Vector Search
- Retrieval must sit behind an application API rather than a SQL endpoint Vertex AI Vector Search
The framing that resolves these: BigQuery vector search is analytics-grade retrieval colocated with your data ; Vertex AI Vector Search is serving-grade retrieval for an application in the request path .
Governance for Retrieval Corpora
De-identify before you embed. An embedding derived from text containing regulated identifiers is a derivative of that data and travels with the same obligations. Run Cloud DLP de-identification during chunking, not after indexing.
Isolate tenants with a pre-filter, not with trust. A multi-tenant corpus in one table must carry a tenant column and pre-filter on it; relying on the generation prompt to "only use tenant A's documents" is not a control.
Keep policy tags on the source columns. Remember that policy-tagged columns are excluded from index stored columns, so plan pre-filtering around unclassified metadata such as tenant ID and document date.
Align regions. The dataset, the Cloud resource connection, and the model endpoint must be regionally compatible; an EU corpus embedded through a US endpoint is a residency finding.
Exam Traps and Antipatterns Summary
Scenario Cue Wrong Answer Correct Approach
"Answers cite the wrong section of a 200-page manual" Increase top_k Chunk smaller with overlap and carry section metadata for citation
"We switched to a newer embedding model; results got worse" Rebuild the vector index Re-embed the entire corpus; mixed-model vectors are not comparable
"Tenant-scoped search returns only two results instead of ten" Raise top_k to 50 Pre-filter on the tenant column so top_k is drawn from the filtered set
"Vector search on our 4 MB reference table ignores the index" File a support case Tables under 10 MB do not populate a vector index; the search falls back to brute force and is still correct
"Rows inserted five minutes ago are missing from results" Force a full index rebuild Unindexed rows are searched by brute force and are not missed
"Semantic search dashboard is slow; buy BI Engine capacity" Add a BI Engine reservation VECTOR_SEARCH and AI.SEARCH are not accelerated by BI Engine
"Extract fields from scanned PDFs before embedding" Load the PDFs as BYTES Object table plus ML.PROCESS_DOCUMENT through Document AI
"Need single-digit millisecond retrieval behind a mobile app" BigQuery vector search Vertex AI Vector Search for serving-grade latency