Study guide
13.2 Groundedness, Relevance, Coherence, and Fluency
Microsoft AI-300: Operationalizing Machine Learning and Generative AI Solutions Exam
Track your progress as you read
Finished sections get a check mark.
1 Introduction & Exam Overview
1.1 Current Exam Facts 1.2 Skills Measured, Format, and How This Guide Is Organized 1.3 Candidate Profile, Azure ML v2, Foundry, and DevOps Tooling
2 Chapter 2: Azure Machine Learning Workspaces, Datastores, Compute, and IAM
2.1 Create and Manage Azure Machine Learning Workspaces 2.2 Create and Manage Datastores 2.3 Create and Manage Compute Targets 2.4 Configure Identity and Access Management for Workspaces
3 Chapter 3: Azure Machine Learning Data Assets, Environments, Components, and Registries
3.1 Create and Manage Data Assets 3.2 Create and Manage Environments 3.3 Create and Manage Components 3.4 Share Assets Across Workspaces with Registries
4 Chapter 4: Infrastructure as Code, GitHub, and Network Isolation for Azure ML
4.1 GitHub Integration and Git Source Control for Machine Learning Projects 4.2 Deploy Workspaces and Resources with Bicep and Azure CLI 4.3 Automate Resource Provisioning with GitHub Actions 4.4 Restrict Network Access to Machine Learning Workspaces
5 Chapter 5: MLflow Tracking, AutoML, Training Jobs, and Hyperparameter Sweeps
5.1 Configure Experiment Tracking with MLflow 5.2 Use Automated Machine Learning to Explore Optimal Models 5.3 Notebooks, Command Jobs, and Training Scripts 5.4 Automate Hyperparameter Tuning
6 Chapter 6: Distributed Training, Training Pipelines, and Job Comparison
6.1 Manage Distributed Training for Large and Deep Learning Models 6.2 Implement Training Pipelines 6.3 Compare Model Performance Across Jobs
7 Chapter 7: Model Registration, Responsible AI, and Lifecycle Management
7.1 Register MLflow Models and Package Feature Retrieval Specs 7.2 Evaluate Models with Responsible AI Principles 7.3 Manage Model Lifecycle and Archiving
8 Chapter 8: Online Endpoints, Batch Endpoints, Rollout, and Rollback
8.1 Deploy Models to Managed Online Endpoints 8.2 Deploy Models to Batch Endpoints 8.3 Test, Troubleshoot, Progressive Rollout, and Safe Rollback
9 Chapter 9: Data Drift, Production Metrics, Retraining, and Alerts
9.1 Detect and Analyze Data Drift 9.2 Monitor Production Metrics and Trigger Retraining or Alerts
10 Chapter 10: Microsoft Foundry Resources, IAM, Networking, and IaC
10.1 Create and Configure Foundry Resources and Project Environments 10.2 Managed Identities and RBAC for Foundry 10.3 Foundry Network Security and Private Networking 10.4 Deploy Foundry Infrastructure with Bicep and Azure CLI
11 Chapter 11: Deploying and Managing Foundation Models
11.1 Serverless API Endpoints and Managed Compute for Foundation Models 11.2 Select Foundation Models for Specific Use Cases 11.3 Model Versioning and Production Deployment Strategies 11.4 Provisioned Throughput Units for High-Volume Workloads
12 Chapter 12: Prompt Design, Variants, and Git Version Control
12.1 Design and Develop Prompts 12.2 Create Prompt Variants and Compare Performance 12.3 Version Control for Prompts with Git
13 Chapter 13: Evaluation Datasets, Quality Metrics, Safety, and Automated Workflows
13.1 Test Datasets and Data Mapping for Evaluation 13.2 Groundedness, Relevance, Coherence, and Fluency 13.3 Risk and Safety Evaluations 13.4 Built-in and Custom Automated Evaluation Workflows
14 Chapter 14: Continuous Monitoring, Performance, Cost, Tracing, and Logging
14.1 Continuous Monitoring in Microsoft Foundry 14.2 Latency, Throughput, Token Use, and Cost Metrics 14.3 Logging, Distributed Tracing, and Production Debugging
15 Chapter 15: Retrieval Tuning, Embeddings, Hybrid Search, and RAG Evaluation
15.1 Tune Similarity Thresholds, Chunk Sizes, and Retrieval Strategies 15.2 Select and Fine-Tune Embedding Models 15.3 Hybrid Search: Semantic and Keyword Retrieval 15.4 RAG Relevance Metrics and A/B Testing
16 Chapter 16: Fine-Tuning Methods, Synthetic Data, and Promotion to Production
16.1 Advanced Fine-Tuning Methods (LoRA, QLoRA, Full) 16.2 Create and Manage Synthetic Data for Fine-Tuning 16.3 Monitor, Optimize, and Promote Fine-Tuned Models
13.2 Groundedness, Relevance, Coherence, and Fluency
Key Takeaways Explain This Topic AI
Domain 4 requires implementing AI quality metrics: groundedness, relevance, coherence, and fluency. In Microsoft Foundry these are built-in evaluators (builtin.groundedness, builtin.relevance, builtin.coherence, builtin.fluency), typically 1–5 Likert with default pass threshold 3.
Groundedness measures whether the response is supported by retrieved context (precision; needs context; query optional but recommended). Relevance measures whether the response addresses the query (query + response). They are not the same score.
Coherence scores logical flow (query + response). Fluency scores grammatical quality and readability (response only). Both are writing-quality metrics independent of factual correctness and currently document English-language responses.
Similarity (LLM, 1–5) and F1/BLEU/GLEU/ROUGE/METEOR (token or n-gram overlap, typically 0–1) require ground_truth. Use them when you have an expected answer; they do not prove the response is grounded in retrieved documents.
LLM-as-judge quality evaluators need a model deployment (initialization_parameters deployment_name). Groundedness Pro (preview) uses Azure AI Content Safety and does not. Judge calls incur inference cost per row.
Last updated: August 2026
Groundedness, Relevance, Coherence, and Fluency
Quick Answer: Groundedness = the response is supported by context (needs retrieved text). Relevance = the response addresses the query . Coherence = logical flow. Fluency = language quality. Scores are typically 1–5 with default pass threshold 3 . Similarity and F1 need ground_truth . LLM judges need a deployment . Groundedness is not relevance.
The official AI-300 skill is implement AI quality metrics (groundedness, relevance, coherence, fluency) . Microsoft Foundry’s catalog implements those as built-in evaluators . You already mapped columns in section 13.1. This section is which metric to trust for which failure.
Quality is a panel, not one number
A claims agent can be fluent (beautiful English) and incoherent (contradicts itself), relevant (talks about deductibles) and ungrounded (invents a $0 deductible the handbook never stated), or grounded (quotes a stale chunk) and irrelevant (ignores the actual question). Promoting a prompt because fluency went from 3 to 5 while groundedness fell from 5 to 2 is a regression. Chapter 12 already said that; here are the evaluators that make it measurable.
Evaluator Measures Required inputs Judge? Output
builtin.groundedness Response supported by context; no fabrication (precision) response + context (query optional but recommended) Yes ( deployment_name ) 1–5; default threshold 3
builtin.groundedness_pro (preview) Strict consistency with context via Azure AI Content Safety query , response , context No service GPT Binary pass/fail
- builtin.relevance Accuracy, completeness, directness versus the query query , response Yes 1–5; threshold 3
- builtin.coherence Logical flow and organization query , response Yes 1–5; threshold 3
- builtin.fluency Grammar, vocabulary, readability response Yes 1–5; threshold 3
builtin.similarity Semantic closeness to ground truth given the query query , response , ground_truth Yes 1–5; threshold 3
builtin.f1_score Token overlap vs ground truth (precision/recall harmonic mean) response , ground_truth No 0–1; default threshold 0.5
Microsoft’s RAG guidance treats groundedness as precision (nothing extra that is not in context) and response completeness (preview) as recall versus ground_truth (nothing critical missing). Retrieval evaluators ( builtin.retrieval , builtin.document_retrieval ) score the search step , not the final sentence — Domain 5 goes deeper on RAG tuning; you still need to know that a bad retrieval score explains a later groundedness failure.
Recommended RAG application combo from the built-in catalog: Retrieval + Groundedness + Relevance + content safety . That is the exam bundle, not fluency alone.
Groundedness is not relevance
Groundedness asks: given this context , did the model stick to it? If the handbook says the waiting period is 90 days and the model says 30, groundedness fails even if 30 days would be a "relevant" sounding benefits answer. If the model recites the 90-day paragraph in perfect prose but the member asked about network providers , relevance fails even if every sentence is in the chunk.
Provide query + response + context for the best groundedness results; Microsoft says query is optional but improves scoring. Concatenate chunks. For agents, context can be omitted when tool-call results are in the trace and the evaluator can extract them — do not rely on that on a dataset evaluation of plain strings. Groundedness Pro is the strict Content Safety variant: no judge deployment , boolean label. Use it when you want a service-backed binary gate; use the GPT judge when you want a 1–5 rubric and a reason string.
Relevance does not take context. It cannot tell you the answer was copied from the wrong document. That is why you run both.
Coherence, fluency, and the English-language caveat
Coherence is whether a reader can follow the argument: order, transitions, no self-contradiction. It needs query and response so the judge can see whether the answer actually addresses the question in a logical sequence. Set evaluation_level to conversation when you care about flow across turns ; all evaluators in that run must support conversation level — you cannot mix a turn-only metric into the same run.
Fluency is surface language quality. It needs only response . A fluent hallucination still scores well. Microsoft notes reliability drops on very short responses (under about 20 tokens) and that both general-purpose evaluators currently support English . Do not certify a Spanish agent with fluency 5 on English judges.
For the best cost/quality balance on LLM judges, Microsoft currently calls out gpt-5-mini . Older local-SDK docs mention gpt-4o-mini . The exam skill is a chat-capable deployment in the project , not a specific SKU you memorize as eternal. No deployment means the quality evaluator cannot run (except Groundedness Pro and the algorithmic overlap metrics).
When ground truth exists: similarity and F1
If adjusters already wrote the right answer, add textual similarity evaluators:
Similarity — LLM semantic match of response to ground truth with respect to the query . Paraphrases can still pass.
F1 — shared tokens; precision is overlap versus the generation, recall is overlap versus the truth.
BLEU / GLEU / ROUGE / METEOR — n-gram overlap family; ROUGE returns precision, recall, and F1. Useful for tight wording, harsh on valid paraphrases.
These metrics require ground_truth . They do not require context. A response can F1-match a stale expert answer that is no longer in the index. Use them as a second opinion next to groundedness, not as a replacement.
Output fields you will see on a judge metric: score , label (pass/fail), reason , threshold , passed . Default quality threshold 3 on the 1–5 scale means 3, 4, and 5 pass. Raise it for high-risk answers; document the change in the evaluation definition so CI does not silently use 3.
Exam scenario
A prompt variant for Contoso’s member chat raises average fluency from 3.4 to 4.6 and coherence from 3.1 to 4.2 on the golden set. Groundedness falls from 4.5 to 2.1 because the new system message says "be helpful even if unsure" and the model fills gaps. Relevance stays at 4.0. You reject the variant. You restore the "answer only from context / I don't know" rule, rerun, and accept only when groundedness is back at or above the threshold and relevance has not collapsed. Fluency was never the ship criterion.
Common trap
Treating relevance as proof of groundedness (or the reverse). A second trap is running groundedness without mapping context , then wondering why scores are noise. A third is using F1 as the only RAG metric because it is "objective" and free of a judge — it needs ground truth and ignores retrieval. A fourth is omitting deployment_name for coherence and assuming safety-style hosted models will judge writing quality. A fifth is mixing conversation-level and turn-only evaluators in one run.