Study guide
15.2 Select and Fine-Tune Embedding Models
Microsoft AI-300: Operationalizing Machine Learning and Generative AI Solutions Exam
Track your progress as you read
Finished sections get a check mark.
1 Introduction & Exam Overview
1.1 Current Exam Facts 1.2 Skills Measured, Format, and How This Guide Is Organized 1.3 Candidate Profile, Azure ML v2, Foundry, and DevOps Tooling
2 Chapter 2: Azure Machine Learning Workspaces, Datastores, Compute, and IAM
2.1 Create and Manage Azure Machine Learning Workspaces 2.2 Create and Manage Datastores 2.3 Create and Manage Compute Targets 2.4 Configure Identity and Access Management for Workspaces
3 Chapter 3: Azure Machine Learning Data Assets, Environments, Components, and Registries
3.1 Create and Manage Data Assets 3.2 Create and Manage Environments 3.3 Create and Manage Components 3.4 Share Assets Across Workspaces with Registries
4 Chapter 4: Infrastructure as Code, GitHub, and Network Isolation for Azure ML
4.1 GitHub Integration and Git Source Control for Machine Learning Projects 4.2 Deploy Workspaces and Resources with Bicep and Azure CLI 4.3 Automate Resource Provisioning with GitHub Actions 4.4 Restrict Network Access to Machine Learning Workspaces
5 Chapter 5: MLflow Tracking, AutoML, Training Jobs, and Hyperparameter Sweeps
5.1 Configure Experiment Tracking with MLflow 5.2 Use Automated Machine Learning to Explore Optimal Models 5.3 Notebooks, Command Jobs, and Training Scripts 5.4 Automate Hyperparameter Tuning
6 Chapter 6: Distributed Training, Training Pipelines, and Job Comparison
6.1 Manage Distributed Training for Large and Deep Learning Models 6.2 Implement Training Pipelines 6.3 Compare Model Performance Across Jobs
7 Chapter 7: Model Registration, Responsible AI, and Lifecycle Management
7.1 Register MLflow Models and Package Feature Retrieval Specs 7.2 Evaluate Models with Responsible AI Principles 7.3 Manage Model Lifecycle and Archiving
8 Chapter 8: Online Endpoints, Batch Endpoints, Rollout, and Rollback
8.1 Deploy Models to Managed Online Endpoints 8.2 Deploy Models to Batch Endpoints 8.3 Test, Troubleshoot, Progressive Rollout, and Safe Rollback
9 Chapter 9: Data Drift, Production Metrics, Retraining, and Alerts
9.1 Detect and Analyze Data Drift 9.2 Monitor Production Metrics and Trigger Retraining or Alerts
10 Chapter 10: Microsoft Foundry Resources, IAM, Networking, and IaC
10.1 Create and Configure Foundry Resources and Project Environments 10.2 Managed Identities and RBAC for Foundry 10.3 Foundry Network Security and Private Networking 10.4 Deploy Foundry Infrastructure with Bicep and Azure CLI
11 Chapter 11: Deploying and Managing Foundation Models
11.1 Serverless API Endpoints and Managed Compute for Foundation Models 11.2 Select Foundation Models for Specific Use Cases 11.3 Model Versioning and Production Deployment Strategies 11.4 Provisioned Throughput Units for High-Volume Workloads
12 Chapter 12: Prompt Design, Variants, and Git Version Control
12.1 Design and Develop Prompts 12.2 Create Prompt Variants and Compare Performance 12.3 Version Control for Prompts with Git
13 Chapter 13: Evaluation Datasets, Quality Metrics, Safety, and Automated Workflows
13.1 Test Datasets and Data Mapping for Evaluation 13.2 Groundedness, Relevance, Coherence, and Fluency 13.3 Risk and Safety Evaluations 13.4 Built-in and Custom Automated Evaluation Workflows
14 Chapter 14: Continuous Monitoring, Performance, Cost, Tracing, and Logging
14.1 Continuous Monitoring in Microsoft Foundry 14.2 Latency, Throughput, Token Use, and Cost Metrics 14.3 Logging, Distributed Tracing, and Production Debugging
15 Chapter 15: Retrieval Tuning, Embeddings, Hybrid Search, and RAG Evaluation
15.1 Tune Similarity Thresholds, Chunk Sizes, and Retrieval Strategies 15.2 Select and Fine-Tune Embedding Models 15.3 Hybrid Search: Semantic and Keyword Retrieval 15.4 RAG Relevance Metrics and A/B Testing
16 Chapter 16: Fine-Tuning Methods, Synthetic Data, and Promotion to Production
16.1 Advanced Fine-Tuning Methods (LoRA, QLoRA, Full) 16.2 Create and Manage Synthetic Data for Fine-Tuning 16.3 Monitor, Optimize, and Promote Fine-Tuned Models
15.2 Select and Fine-Tune Embedding Models
Key Takeaways Explain This Topic AI
Query embeddings and index embeddings must come from the same model, version, and dimensions. Azure AI Search will not meaningfully compare vectors from mixed models in one vector field.
You cannot in-place upgrade text-embedding-ada-002 to text-embedding-3-large. Changing the embedding model requires generating new vectors and re-indexing every document.
Foundry-sold Azure OpenAI embeddings: text-embedding-3-large (up to 3,072 dimensions, 8,192-token input), text-embedding-3-small (1,536 dimensions), and text-embedding-ada-002 (1,536). Third-generation models support a dimensions parameter to shrink storage.
Fine-tune or replace an embedding model only when domain vocabulary (SKUs, ICD codes, internal acronyms) is poorly represented by general models. Chat SFT in Foundry is not a substitute for re-embedding the corpus.
Azure OpenAI embeddings use cosine similarity in Azure AI Search. Configure the vector field’s similarity metric and the index vectorizer to the same deployed embedding endpoint used at query time.
Last updated: August 2026
Select and Fine-Tune Embedding Models
Quick Answer: An embedding is a dense vector that represents meaning. In Microsoft Foundry RAG, the same embedding model , version , and dimensions must vectorize documents and queries. text-embedding-3-large is the strongest Azure OpenAI embedder (up to 3,072 dimensions, 8,192 -token input). You cannot mix models in one vector field , and you must re-index after any model change. Fine-tune a custom embedder only when domain jargon is far from general web text.
Domain 5 continues with select and fine-tune embedding models for domain-specific use cases . Exam AI-300 expects you to treat embeddings as production infrastructure: a Foundry deployment, an Azure AI Search vectorizer , and an index field whose dimensions match the model output.
What an embedding is in this stack
You send text to an embeddings API (Azure OpenAI in Foundry, or another catalog embedder). The service returns an array of floats. Nearby vectors mean similar meaning, which is why HNSW can find “waiting period” near “elimination period” even when the words differ. Integrated vectorization in Azure AI Search calls a configured vectorizer at index time (skillset) and at query time ( vectorQueries.kind = text ), so application code does not have to call the embeddings API itself.
Hard constraints from current Microsoft docs:
Current Azure OpenAI embedding models accept up to 8,192 tokens per input .
A single embeddings request can send at most 2,048 input strings, and the sum of tokens across the batch must stay under 300,000 or the call returns HTTP 400.
The model value is your deployment name , not necessarily the base model ID.
Microsoft Entra ID with Cognitive Services OpenAI User is the recommended auth path; keys are supported but not the exam’s preferred production pattern.
Catalog selection: dimensions, language, domain
Foundry Models sold by Azure include three Azure OpenAI embedders. Microsoft states that text-embedding-3-large is the latest and most capable, and that you cannot upgrade between embedding models —moving from ada-002 to 3-large means generating new embeddings .
- Model Max input tokens Default output dimensions Notes
- text-embedding-ada-002 v2 8,192 1,536 Previous generation; no dimensions shrink
- text-embedding-3-small 8,192 1,536 Third generation; supports dimensions ; cheaper
- text-embedding-3-large 8,192 3,072 Strongest OpenAI embedder in the catalog; supports dimensions
OpenAI’s published MTEB (English) and MIRACL (multilingual retrieval) averages improve from ada-002 → 3-small → 3-large (MIRACL 31.4 / 44.0 / 54.9; MTEB 61.0 / 62.3 / 64.6). If the corpus is mixed-language benefits content, 3-large’s MIRACL lift is the reason to pay for more dimensions—not because “larger is always required for English FAQs.”
Third-generation models accept a dimensions parameter so you can store fewer than 3,072 or 1,536 floats. Microsoft notes that even reduced 3-large vectors can still beat ada-002’s 1,536-d MTEB score. Smaller dimensions cut Azure AI Search storage and HNSW memory. The vector field’s dimensions in the index schema must equal the vectors you write. You cannot store 3,072-d and 1,536-d rows in the same field.
The Foundry catalog also lists partner embedding models (for example Cohere embed families) and multimodal embedders. Use a partner model when you need that vendor’s language coverage or image+text vectors. The rule does not change: one model per vector field , query-side identical to index-side.
Selection checklist:
Language: English-only FAQs can start on 3-small; cross-lingual retrieval should compare 3-large (or a multilingual partner model) on a labeled set.
Domain: Product SKUs and error codes still need keyword/hybrid even with a strong embedder (section 15.3).
Cost and latency: 3-large at 3,072-d is the quality default; shrink dimensions before you jump to a custom model.
Similarity metric: Azure OpenAI embeddings are trained for cosine . Set the Azure AI Search vector profile to cosine , not Euclidean, unless you have a non-OpenAI model that specifies otherwise.
Same model for query and index — no mixing
Azure AI Search can query up to 10 vector fields in one vectorQueries item, but Microsoft requires that each field contain embeddings from the same embedding model , and that the query vector is generated from that same model. Mixing ada-002 document vectors with a 3-large query is not “close enough”: the spaces are incompatible, ranks are noise, and you may even fail dimension checks.
Operational implications:
Record the embedding deployment name , model ID , version , and dimensions in the index metadata and in Git.
Point the index vectorizer at that deployment so query-time kind: text cannot silently drift.
When you change any of those four facts, re-embed the entire corpus and rebuild or reload the vector field . There is no rolling mixed-model index.
Do not “A/B two embedding models” inside one field. Stand up a second index (or a second vector field populated entirely by the candidate model), run the same query set, and compare nDCG / recall@k (section 15.4).
Exam scenario: A team upgrades a Foundry embedding deployment from text-embedding-ada-002 to text-embedding-3-large and only re-embeds new SharePoint files. Old vectors stay 1,536-d ada space; new vectors are 3,072-d. Queries encoded with 3-large cannot be compared to the old rows, and the field dimensions no longer match. The correct change control is: create or update the vector field for 3,072 (or a chosen reduced dimension), batch-embed all documents, reset the indexer, and point both the skillset and the query vectorizer at the new deployment.
When to fine-tune — and what that actually means
Foundry fine-tuning UIs and SFT/DPO/RFT jobs target chat and reasoning models , not the Azure OpenAI embedding trio. Do not tell the exam that you “fine-tune ada-002 like GPT-4o-mini” in the same wizard.
Fine-tune or customize an embedding model when a general embedder systematically fails on domain vocabulary that never appeared in web-scale training: internal SKU syntax, clinical codes, legal citations, or a private language mix. Typical paths:
Stay with a catalog model first. Better chunking, hybrid search, and 3-large often close the gap without a custom embedder.
Shrink or grow dimensions on 3-small/3-large as a cheaper experiment than training.
Train a domain embedder (contrastive query–passage pairs, or a community sentence-transformer) and host it on managed compute or a supported catalog/partner endpoint. Then treat it as the new source of truth for that vector field.
Partner embedders (Cohere and others in the catalog) when their domain or language fit is documented and you accept a second bill.
If you do train or swap, the blast radius is the whole index . Query-time embedding must switch on the same day as the re-index. Keep the old index until the new one beats it on a frozen eval set. Fine-tuning the generator (LoRA on a chat model, Chapter 16) will not fix neighbors that were never close in vector space.
Common trap: Deploying text-embedding-3-large for queries while the indexer skill still calls ada-002 “until we have time to rebuild.” Every query then searches the wrong space. A second trap is stuffing two models’ vectors into one field by padding 1,536-d arrays with zeros to look like 3,072-d — the geometry is meaningless.
Lock the embedding identity in Bicep alongside the Search index: model name, version, dimensions, cosine metric, and vectorizer resource ID. That is the GenAIOps control the exam is after.