5.1 Embeddings & Vector Similarity Search in Cosmos DB

Exam AI-200: Developing AI Cloud Solutions on Azure Exam

Track your progress as you read

Finished sections get a check mark.

1 Introduction & Exam Overview

1.1 Current Exam Facts 1.2 Audience Profile & Study Path

2 Chapter 1: Container Registry & App Service Hosting

1.1 ACR: Build, Store, Version & Manage Images 1.2 ACR Tasks: Build & Run Images 1.3 Deploy Containers to Azure App Service

3 Chapter 2: Azure Container Apps & KEDA Scaling

2.1 Azure Container Apps: Environments, Config & Revisions 2.2 KEDA Event-Driven Scaling in Container Apps

4 Chapter 3: AKS Manifests & Container Troubleshooting

3.1 Deploy & Manage Apps on AKS with Manifests 3.2 Monitor & Troubleshoot AKS and Container Apps

5 Chapter 4: Cosmos DB for NoSQL Fundamentals

4.1 Cosmos DB for NoSQL: SDK Connect & Queries 4.2 Optimize RUs: Indexing Policies & Consistency Levels

6 Chapter 5: Cosmos DB Vectors & Change Feed

5.1 Embeddings & Vector Similarity Search in Cosmos DB 5.2 Change Feed Processor for New/Updated Items

7 Chapter 6: Azure PostgreSQL Foundations for AI

6.1 Azure Database for PostgreSQL: Connect & Query with SDKs 6.2 Schema Design, Data Types & Indexing Strategies 6.3 pgvector Indexing: Latency & Compute Overhead

8 Chapter 7: PostgreSQL Vector Workloads & RAG

7.1 Configure Compute, Memory & Storage for Vector Workloads 7.2 Vector Search, Semantic Retrieval & RAG with Metadata Filters 7.3 Connection Optimization for Throughput & Latency

9 Chapter 8: Azure Managed Redis for AI Solutions

8.1 Azure Managed Redis: Caching, Expiration & Invalidation 8.2 Redis Vector Indexing for Similarity Search

10 Chapter 9: Service Bus & Event Grid for AI Pipelines

9.1 Service Bus Queues & Message Processing 9.2 Topics, Subscriptions & Dead-Letter Queues 9.3 Event Grid: Custom Events & Filters 9.4 Event Grid Retries & Event-Driven Workflows

11 Chapter 10: Azure Functions for Serverless AI Back Ends

10.1 Azure Functions: Triggers & Bindings 10.2 Build Serverless APIs with Functions 10.3 Configure & Deploy Function Apps

12 Chapter 11: Key Vault & App Configuration

11.1 Azure Key Vault: Secret Storage & Retrieval 11.2 Key Vault Secret Rotation Patterns 11.3 Azure App Configuration for App Settings

13 Chapter 12: OpenTelemetry & KQL Observability

12.1 Distributed Tracing with OpenTelemetry SDKs 12.2 Correlating Traces into Azure Monitoring 12.3 Write KQL Queries to Analyze Logs 12.4 KQL for Metrics & Troubleshooting Scenarios

5.1 Embeddings & Vector Similarity Search in Cosmos DB

Key Takeaways Explain This Topic AI

Azure Cosmos DB for NoSQL stores document embeddings in the same item as source text, enabling RAG retrieval without a separate vector database

A container vector embedding policy declares the vector path, float32 data type, dimensions, and distance function (cosine, euclidean, or dotproduct)

DiskANN is the preferred vector index for large-scale approximate nearest neighbor search; flat and quantizedFlat suit smaller or exact-match scenarios

Similarity search uses ORDER BY VectorDistance(vectorPath, @queryVector) with TOP N to return the closest chunks for grounding a language model

Exclude embedding property paths from the default indexing policy so vectors use only the specialized vector index and do not waste request units

Last updated: July 2026

Why Vector Search Matters for AI-200

Many Azure AI solutions need semantic retrieval : find documents that mean the same thing as a user question, not only documents that share exact keywords. That capability depends on vector embeddings —numeric arrays produced by an embedding model (for example Azure OpenAI text-embedding-3-small or text-embedding-3-large ) that place similar meanings near each other in multi-dimensional space.

Azure Cosmos DB for NoSQL can store those embeddings next to the original chunk text, metadata, and partition key in a single JSON item. You then run vector similarity search in the same database that already powers your operational reads and writes. For the exam, treat this as the data-plane pattern for retrieval-augmented generation (RAG) : embed the query, search for nearest neighbors, pass retrieved chunks to a chat or completion model, and answer with grounded context.

Storing vectors in Cosmos DB reduces synchronization work. When a product description, support article, or meeting transcript changes, one upsert updates both the human-readable fields and the embedding. You avoid a dual-write pipeline to a separate vector store—unless product requirements force that split.

Embedding Shape Inside a Cosmos DB Item

A typical RAG chunk item looks like this conceptually:

Partition key such as /tenantId or /sourceId for scale-out and RU isolation
Content fields : title , content , sourceUri , chunkIndex , timestamps

Embedding field : contentVector — an array of floating-point numbers whose length matches the embedding model’s dimensions (for example 1536 for many OpenAI small models)

The embedding must be generated before or during write. Cosmos DB does not invent vectors for you; your application (or a change-feed worker covered in the next section) calls an embedding API and persists the result.

Consistency Rules You Must Remember

Dimensions are fixed per vector path. Every item that uses /contentVector must supply an array of the same length declared in the container’s vector embedding policy.

Distance function must match how the model was trained. Cosine similarity is common for normalized text embeddings; euclidean and dot product are also supported when you configure them explicitly.

Null or missing vectors are not searchable. Items without a populated vector path simply will not rank in similarity results for that path.

Vector Embedding Policy

When you create (or later update, where supported) a container for vector workloads, you define a vector embedding policy . That policy tells the engine which JSON paths hold vectors and how to interpret them.

Policy setting What it controls Exam tip
path JSON path to the vector property (for example /contentVector ) Must match the property you write on items
dataType Usually float32 for embeddings Keep consistent with SDK serialization
dimensions Length of each embedding array Must match the embedding model output size

distanceFunction cosine , euclidean , or dotproduct Choose once; queries use VectorDistance against this metric

You can declare multiple vector paths on one container when different properties store different embedding spaces (for example a title embedding and a body embedding). Each path needs its own dimensions and distance function settings that match how those vectors were produced.

Vector Index Types: Flat, Quantized Flat, and DiskANN

Cosmos DB for NoSQL offers specialized vector indexes . The index type is a deliberate trade-off among recall, latency, storage, and scale.

Index type Behavior Best fit

flat Exact nearest neighbor over stored vectors Small collections, highest accuracy when RU/latency budget allows

quantizedFlat Compressed vectors with flat-style search Medium scale when you need lower memory/cost than full flat

diskANN Approximate nearest neighbor (ANN) using Microsoft Research DiskANN / Vamana-style graph indexing with SSD-backed scale Large corpora, production RAG, high QPS with strong recall

DiskANN keeps quantized (compressed) representations efficiently accessible while using SSD-backed structures so the index can grow far beyond what an in-memory-only index would allow. It supports ongoing inserts, updates, and deletes without forcing a full rebuild for every change—critical for operational AI apps that continuously ingest content.

For AI-200 scenario questions that mention millions of chunks, low latency, and cost-effective approximate search, prefer DiskANN . Reserve flat for demos, tiny knowledge bases, or cases that explicitly require exact search.

Indexing Policy Hygiene

Exclude the vector property from the default (non-vector) index . Example concept: add /contentVector/? to excluded paths in the indexing policy. Vectors already participate in the specialized vector index; also indexing them as ordinary arrays wastes storage and request units (RUs) without helping VectorDistance queries.

Executing Similarity Search for Semantic Retrieval

At query time:

Call your embedding model with the user question (and optional conversation context).

Pass the resulting float array as a parameterized value @queryVector .

Run a NoSQL query that ranks by distance and limits results.

A representative pattern: SELECT TOP 5 c.title, c.content, c.sourceUri
FROM c
WHERE c.tenantId = @tenantId
ORDER BY VectorDistance(c.contentVector, @queryVector)

Key exam ideas:

VectorDistance returns a distance score according to the container’s configured distance function; lower distance means closer (more similar) for typical cosine/euclidean setups used in retrieval.

TOP N (or an equivalent limit) is how you implement “retrieve the top-k chunks” for RAG.

Filters and vector ranking compose. You can combine equality or range filters (tenant, language, product line, ACL tags) with vector ordering so retrieval stays tenant-safe and relevant.

Cross-partition queries may be required when the semantic corpus spans many partition key values; design partition keys so hot tenants or sources stay balanced.

RAG Pipeline Role

In a full Azure AI solution, Cosmos DB similarity search is usually the retriever step:

User asks a question in your app or Azure AI Foundry / OpenAI chat flow.

App embeds the question.

Cosmos DB returns top-k chunks via DiskANN-backed VectorDistance.

App builds a grounded prompt (system instructions + retrieved chunks + question).

Chat model generates the answer; optionally log citations from sourceUri .

Because vectors live with documents, hybrid patterns —keyword filters, metadata boosts, or later hybrid full-text + vector ranking where available—can sit in the same query surface without shipping raw text to a second store solely for search.

Operational Design Choices

When to embed online vs offline. Batch jobs can embed historical corpora. Online paths embed on ingest. Many production designs use the change feed (next section) so new or updated items automatically receive embeddings and become searchable without blocking the write API on the embedding call.

Dimension and model upgrades. Changing embedding models often changes dimensions. That typically means a new vector path or a carefully planned re-embed and re-index of the corpus—not a silent in-place resize of an existing policy path.

RU and latency. Vector search costs RUs like other queries. DiskANN and sensible TOP N keep retrieval affordable. Oversized TOP N , missing filters, and poorly chosen partition keys drive cost and latency up.

Security. Treat embeddings as sensitive derivatives of content. Apply the same RBAC, private networking, and encryption controls you use for the source documents. Never assume “it’s just floats” and skip tenant isolation in filters.

Exam Scenario Checklist

If a question describes storing product manuals and answering support questions with grounded GPT responses, look for: container vector embedding policy, DiskANN (or appropriate index), VectorDistance + TOP , same-item storage of text + vector, and exclusion of the vector path from the default index. If the scenario emphasizes exact matches on tiny datasets, flat may appear as the correct index choice. If it emphasizes continuous ingest at scale with low latency ANN, DiskANN is the signal answer.