7.2 Vector Search, Semantic Retrieval & RAG with Metadata Filters

Exam AI-200: Developing AI Cloud Solutions on Azure Exam

Track your progress as you read

Finished sections get a check mark.

1 Introduction & Exam Overview

1.1 Current Exam Facts 1.2 Audience Profile & Study Path

2 Chapter 1: Container Registry & App Service Hosting

1.1 ACR: Build, Store, Version & Manage Images 1.2 ACR Tasks: Build & Run Images 1.3 Deploy Containers to Azure App Service

3 Chapter 2: Azure Container Apps & KEDA Scaling

2.1 Azure Container Apps: Environments, Config & Revisions 2.2 KEDA Event-Driven Scaling in Container Apps

4 Chapter 3: AKS Manifests & Container Troubleshooting

3.1 Deploy & Manage Apps on AKS with Manifests 3.2 Monitor & Troubleshoot AKS and Container Apps

5 Chapter 4: Cosmos DB for NoSQL Fundamentals

4.1 Cosmos DB for NoSQL: SDK Connect & Queries 4.2 Optimize RUs: Indexing Policies & Consistency Levels

6 Chapter 5: Cosmos DB Vectors & Change Feed

5.1 Embeddings & Vector Similarity Search in Cosmos DB 5.2 Change Feed Processor for New/Updated Items

7 Chapter 6: Azure PostgreSQL Foundations for AI

6.1 Azure Database for PostgreSQL: Connect & Query with SDKs 6.2 Schema Design, Data Types & Indexing Strategies 6.3 pgvector Indexing: Latency & Compute Overhead

8 Chapter 7: PostgreSQL Vector Workloads & RAG

7.1 Configure Compute, Memory & Storage for Vector Workloads 7.2 Vector Search, Semantic Retrieval & RAG with Metadata Filters 7.3 Connection Optimization for Throughput & Latency

9 Chapter 8: Azure Managed Redis for AI Solutions

8.1 Azure Managed Redis: Caching, Expiration & Invalidation 8.2 Redis Vector Indexing for Similarity Search

10 Chapter 9: Service Bus & Event Grid for AI Pipelines

9.1 Service Bus Queues & Message Processing 9.2 Topics, Subscriptions & Dead-Letter Queues 9.3 Event Grid: Custom Events & Filters 9.4 Event Grid Retries & Event-Driven Workflows

11 Chapter 10: Azure Functions for Serverless AI Back Ends

10.1 Azure Functions: Triggers & Bindings 10.2 Build Serverless APIs with Functions 10.3 Configure & Deploy Function Apps

12 Chapter 11: Key Vault & App Configuration

11.1 Azure Key Vault: Secret Storage & Retrieval 11.2 Key Vault Secret Rotation Patterns 11.3 Azure App Configuration for App Settings

13 Chapter 12: OpenTelemetry & KQL Observability

12.1 Distributed Tracing with OpenTelemetry SDKs 12.2 Correlating Traces into Azure Monitoring 12.3 Write KQL Queries to Analyze Logs 12.4 KQL for Metrics & Troubleshooting Scenarios

7.2 Vector Search, Semantic Retrieval & RAG with Metadata Filters

Key Takeaways Explain This Topic AI

Store embeddings in a pgvector column alongside chunk text and structured metadata used for filtering

Run vector similarity search with distance operators and ANN indexes (HNSW or IVFFlat) to retrieve top-k semantically related chunks

RAG follows retrieve-then-generate: query embedding → filtered similarity search → prompt grounding → model completion

Metadata filters in WHERE clauses (tenant, product, language, time window) constrain the candidate set before or with ranking by vector distance

Hybrid retrieval that combines semantic similarity and business predicates is the production pattern AI-200 expects for grounded answers

Last updated: July 2026

7.2 Vector Search, Semantic Retrieval & RAG with Metadata Filters

Quick Answer: Implement RAG on Azure Database for PostgreSQL by storing chunk text + embeddings + metadata, retrieving top-k neighbors with vector similarity search, applying WHERE metadata filters for tenancy and business constraints, then sending only those chunks to the generative model. This retrieve-then-generate flow is the core pattern for grounded AI answers.

This section maps directly to the skill: run vector similarity search, including storing embeddings, semantic retrieval, and implementing retrieval-augmented generation (RAG) patterns by using metadata filters .

Storing Embeddings in PostgreSQL

A typical RAG table design keeps retrieval artifacts together:

• Column Purpose
• id Stable chunk identifier
• content Source text chunk sent to the LLM
• embedding vector(n) embedding from the embedding model
• document_id Parent document key
• tenant_id / workspace_id Isolation filter
• category , language , product_sku Business metadata filters
• created_at / effective_on Time-window filters
• source_uri Citation / grounding reference

Enable the vector extension on Azure Database for PostgreSQL Flexible Server, choose a dimension that matches your embedding model, and insert embeddings in the same transaction path that persists chunk text. Keeping metadata beside the vector lets one query enforce security and relevance without a second store round-trip.

Embedding Lifecycle Notes

Model upgrades change dimensions or geometry—plan a parallel column or table and cut over after backfill.

Chunking strategy (size, overlap) affects both storage volume and retrieval quality.

Normalization may be required depending on distance metric (for example, cosine versus inner product workflows).

Idempotent re-embed jobs should update vectors without duplicating chunks.

Vector Similarity Search Mechanics

Semantic retrieval finds chunks whose embeddings are nearest to a query embedding produced from the user question with the same embedding model.

Common pgvector distance styles include:

• Operator / idea Similarity notion Typical use
• Cosine distance ( <=> ) Angle between vectors Text embeddings, normalized spaces
• L2 distance ( <-> ) Euclidean distance Spatial-like embedding spaces
• Inner product ( <#> ) Dot-product ranking Some normalized model setups

Queries generally ORDER BY distance to the query vector and LIMIT to top-k (for example, 3–20 chunks). ANN indexes make this practical at scale:

IVFFlat — partitions vectors into lists; good build characteristics; tune lists/probes for recall vs latency.

HNSW — graph-based ANN; strong query latency/recall trade-offs for many production RAG systems; higher build cost and memory.

Exact scans are fine for tiny labs; production corpora need ANN indexes plus monitoring of recall quality.

Semantic Retrieval Versus Keyword Search

Semantic retrieval matches meaning: “reset MFA” can retrieve a chunk titled “multi-factor authentication recovery” even without shared keywords. Keyword or full-text search matches tokens. Many Azure solutions combine both (hybrid), but AI-200’s PostgreSQL vector emphasis is the embedding path: embed query → similarity search → return chunks.

Semantic retrieval quality depends on chunk boundaries, embedding model choice, top-k, and—critically— filters that remove irrelevant or unauthorized rows before generation.

RAG Pattern: Retrieve-Then-Generate

Retrieval-augmented generation (RAG) reduces hallucination by grounding the model on private or domain data:

Accept the user question and optional application filters (tenant, locale, product).

Embed the question with the same model used at index time.

Retrieve top-k chunks via vector similarity search, constrained by metadata predicates.

Assemble a prompt that includes instructions plus retrieved evidence (and citations).

Generate the answer with Azure OpenAI or another chat/completions model.

Return the answer with source references from source_uri / document metadata.

The database’s job is step 3 (and storing the corpus for step 3). The generative model never needs write access to the vector table for ordinary Q&A. User question

• → embedding model
• → SQL: WHERE metadata filters + ORDER BY embedding <=> :query LIMIT k
• → prompt packer
• → generative model
• → grounded answer + citations
• Metadata Filters: The Production Differentiator

Unfiltered ANN search over a multi-tenant corpus is a security and quality bug. Metadata filters encode business rules in SQL WHERE clauses so retrieval cannot cross tenants or pull expired policy docs.

Examples of filter dimensions:

• Tenancy / ACL — tenant_id = @currentTenant
• Product scope — product_sku = ANY(@skus)
• Language — language = 'en'
• Freshness — effective_on <= now() AND (expires_on IS NULL OR expires_on > now())
• Sensitivity — classification <= @userClearance
• Collection — kb_id = @selectedKnowledgeBase
• Pattern: Filter + Vector Rank
• A canonical shape looks like: SELECT id, content, source_uri,
• embedding <=> @query_embedding AS distance
• FROM kb_chunks
• WHERE tenant_id = @tenant
• AND language = @lang
• AND product_sku = @sku
• ORDER BY embedding <=> @query_embedding
• LIMIT @k;

The WHERE clause narrows candidates; the ORDER BY distance clause ranks remaining rows semantically; LIMIT caps context sent to the LLM. This is the RAG pattern “by using metadata filters” that the exam describes.

Why Filters Belong in the Database Layer

Filtering only in application memory after fetching global top-k is fragile: you may retrieve k neighbors from the wrong tenant and end with zero usable rows, or worse, leak content if the app filter is buggy. Pushing predicates into PostgreSQL enforces isolation next to the vectors and lets the planner use metadata indexes (B-tree/GIN) together with the vector index strategy your extension supports.

Designing Filters for Recall and Safety

• Goal Technique
• Tenant isolation Mandatory equality filter on tenant/workspace key
• Higher precision Add product/category predicates before raising k
• Higher recall Relax optional filters; increase k; tune ANN probes/ef_search
• Citation quality Select source fields with content; keep chunk text intact
• Prompt budget Keep k small enough for the model context window

Tune k as a product decision: too small misses evidence; too large wastes tokens and can dilute the prompt. Filters often improve answer quality more than simply increasing k.

End-to-End Example Scenario

An Azure AI chatbot for Contoso support embeds each knowledge article chunk into PostgreSQL. When a Contoso Canada agent asks about refund windows for SKU Contoso-Bike-2026 in French:

The app embeds the question.

SQL applies tenant_id = contoso , region = ca , language = fr , product_sku = Contoso-Bike-2026 .

Vector search ranks matching chunks.

Top chunks ground Azure OpenAI.

The reply cites article URLs from source_uri .

Without metadata filters, the same similarity search might surface US English policy chunks or another tenant’s data—semantically “close” but operationally wrong.

Operational Practices for RAG Corpora

Re-embed when the embedding model changes; keep dimension metadata documented.

Monitor empty-result rates after filters (over-filtering) versus cross-collection contamination (under-filtering).

Use transactions for chunk+embedding writes; avoid orphan vectors without content.

Version prompts separately from the vector schema.

Load-test filtered ANN queries, not only unfiltered lab queries.

Exam Focus

When a scenario says answers are ungrounded, look for missing retrieval. When answers leak across customers, look for missing WHERE tenancy filters. When results are irrelevant despite embeddings, inspect chunking, model mismatch, top-k, or missing business metadata predicates. The winning architecture is consistently store embeddings → filtered vector similarity search → generate .