Study guide
1.2 Grounding Data Evaluation: Accuracy, Relevance, Timeliness, Cleanliness & Availability
Microsoft Certified: Agentic AI Business Solutions Architect Expert AB-100
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Requirements Analysis & Grounding Data
1.1 Assessing Agents for Task Automation, Analytics & Decision-Making 1.2 Grounding Data Evaluation: Accuracy, Relevance, Timeliness, Cleanliness & Availability 1.3 Structuring Enterprise Business Data for AI System Interoperability
2 Chapter 2: AI Strategy, Cloud Adoption Framework & CoE
2.1 Cloud Adoption Framework (CAF) for Azure AI Strategy & Roadmap 2.2 Multi-Agent Architecture: Microsoft 365 Copilot, Copilot Studio & Foundry 2.3 Microsoft AI Center of Excellence (CoE) & Solution Governance 2.4 Developing Use Cases for Prebuilt Agents & the Microsoft Agent Catalog 2.5 Solution Rules & Constraints for Copilot Studio, Microsoft Foundry & Foundry Tools
3 Chapter 3: Model Selection, Prompt Engineering & Dynamics AI
3.1 Model Selection Strategy, Custom Models & Small Language Models (SLMs) 3.2 Enterprise Prompt Engineering Guidelines & Prompt Libraries 3.3 Cross-App AI Strategy for Multi-App Dynamics 365 Solutions
4 Chapter 4: ROI, TCO, Build vs Buy & Model Routing
4.1 ROI Criteria & Total Cost of Ownership (TCO) for AI Solutions 4.2 Comprehensive Financial ROI Analysis for Business Processes 4.3 Build vs Buy vs Extend Architectural Decision Framework 4.4 Dynamic Model Router Architecture & Cost-Performance Optimization
5 Chapter 5: Dynamics 365 AI & Contact Center Design
5.1 Business Terms & Copilot Customization in Customer Experience and Service 5.2 Connectors & Data Flow Design for Copilot in Dynamics 365 Sales 5.3 Agent Integration with Dynamics 365 Contact Center Channels
6 Chapter 6: Agent Archetypes & Copilot Studio Core
6.1 Designing Task Agents, Autonomous Agents & Prompt-and-Response Agents 6.2 Copilot Studio Topic Design, Trigger Phrases & Graceful Fallbacks 6.3 Orchestration Selection: Standard NLP vs CLU vs Generative AI Orchestration
7 Chapter 7: Copilot Studio Advanced Actions & Power Platform
7.1 Designing Agent Flows & Prompt Actions in Copilot Studio 7.2 Power Apps Canvas App Integration, Generative Pages & Agent Feeds 7.3 Power Platform Well-Architected Framework for AI Workloads 7.4 Proposing Microsoft Power Platform AI Features & the AI Hub
8 Chapter 8: Agent Extensibility, MCP & Computer Use
8.1 Microsoft Foundry Custom Models & Foundry Tools Selection 8.2 Agent Extensibility via Model Context Protocol (MCP) in Copilot Studio 8.3 Computer Use Automation & Web UI Interactions in Copilot Studio
9 Chapter 9: Advanced Behaviors & Prebuilt App Orchestration
9.1 Agent Behaviors: Multi-Step Reasoning, Voice Mode & M365 Agent Optimization 9.2 Orchestrating Prebuilt AI in Dynamics 365 F&O and M365 Copilot for Sales/Service 9.3 Orchestrating Prebuilt Copilot AI Features in Dynamics 365 Customer Experience & Service 9.4 Proposing Microsoft 365 Agents for Business Scenarios 9.5 Adding Knowledge Sources to In-App Help & Guidance in Dynamics 365 Finance & Supply Chain
10 Chapter 10: Telemetry, Monitoring & Agent Tuning
10.1 Monitoring Architectures: Azure Monitor, Application Insights & Studio Analytics 10.2 Telemetry Data Interpretation, Performance Metrics & Latency Tuning 10.3 User Feedback Analysis, Backlog Prioritization & Continuous Improvement
11 Chapter 11: Testing Strategy & Model Validation
11.1 Agent Testing Frameworks, Quality Metrics & Groundedness Benchmarks 11.2 Custom AI Model Validation, Safety Criteria & Regression Testing 11.3 Copilot-Assisted Test Case Generation & Cross-Dynamics End-to-End Testing 11.4 Validating Effective Copilot Prompt Practices
12 Chapter 12: ALM for Grounding Data & Copilot Studio Agents
12.1 ALM for Grounding Data, Vector Indexes & Knowledge Synchronization 12.2 ALM for Copilot Studio Agents, Connectors & Solution Packaging 12.3 Environment Strategy & Automated Deployment Pipelines for Studio Agents
13 Chapter 13: ALM for Foundry Agents & Dynamics 365 AI
13.1 ALM for Microsoft Foundry Agents Service & Model Registries 13.2 ALM for Custom AI Models, Fine-Tuning Artifacts & Deployment Slots 13.3 ALM for Dynamics 365 AI in Finance, Supply Chain & Customer Engagement
14 Chapter 14: Agent Identity, Security & Access Controls
14.1 Agent Identity Architecture, Microsoft Entra ID & Authentication Flows 14.2 Access Controls, Role-Based Security & Permissions on Grounding Data 14.3 Secure Data Processing & Grounding Data Isolation
15 Chapter 15: Model Security & Threat Mitigation
15.1 Model Security Architecture & Defense Against Vulnerabilities 15.2 Mitigating Prompt Manipulation, Jailbreaks & Indirect Prompt Injections 15.3 Azure AI Content Safety, Prompt Shields & Guardrails
16 Chapter 16: Responsible AI, Governance, Residency & Audit
16.1 Microsoft Responsible AI Standard: Principles, Assessment & Implementation 16.2 Data Residency, Sovereign Boundaries & Cross-Border Compliance 16.3 Comprehensive Audit Trails, Model Decision Logging & Microsoft Purview 16.4 Designing Governance for Agents: Environment Strategy, Data Policies & Agent Inventory
1.2 Grounding Data Evaluation: Accuracy, Relevance, Timeliness, Cleanliness & Availability
Key Takeaways Explain This Topic AI
Grounding data readiness must be evaluated across five core architectural dimensions: Accuracy (authoritative source of truth), Relevance (semantic precision and token economy), Timeliness (data freshness SLAs), Cleanliness (noise stripping and structural chunking), and Availability (latency and uptime resiliency).
The RAG Triad—Context Relevance, Groundedness (Faithfulness), and Answer Relevance—serves as the primary quantitative metric framework for evaluating grounding quality and eliminating hallucination in production agents.
Timeliness trade-offs mandate choosing between cached vector indexes (sub-second query speed, batch synchronization latency), real-time API connectors (zero data lag, higher query latency and rate limit risks), or event-driven index synchronization via Azure Event Grid.
Stale grounding data in policy agents is remediated using Azure AI Search scoring profiles with exponential date-decay functions, strict OData pre-filtering on effective date metadata, and automated cache invalidation upon document updates.
Last updated: September 2026
Grounding Data Evaluation: Accuracy, Relevance, Timeliness, Cleanliness & Availability
Quick Answer: Grounding data is the bedrock of enterprise agent reliability. The AB-100 exam evaluates your ability to audit and score grounding data across five dimensions: Accuracy (golden record authority), Relevance (semantic alignment and context budget), Timeliness (freshness SLAs and cache strategies), Cleanliness (structural normalization and noise reduction), and Availability (rate limits and uptime SLAs). Quantitative evaluation relies on the RAG Triad framework.
An agent is only as intelligent as the data grounding its reasoning. In enterprise architectures, the primary root cause of model hallucinations, non-deterministic errors, and regulatory compliance breaches is not model capability, but ungoverned, stale, noisy, or unauthenticated grounding data .
As a Microsoft Solutions Architect, you must systematically evaluate, score, and remediate enterprise knowledge sources before connecting them to Microsoft Copilot Studio or Azure AI Foundry agents.
- 1. The 5 Core Dimensions of Grounding Data Readiness +---------------------------------------+
- | GROUNDING DATA READINESS |
- +---------------------------------------+
- |
- +----------------+---------------+---------------+----------------+
- | | | | |
- v v v v v
- [ ACCURACY ] [ RELEVANCE ] [ TIMELINESS ] [ CLEANLINESS ] [ AVAILABILITY ]
- Golden records - Vector match - Freshness SLA - Markdown norm - 99.99% Uptime
- Conflict rules - Token budget - Real-time vs - Noise removal - TPM / RPM limits
- Source of truth- Metadata filter cached index - Parent-child - VNet isolation
- 1.1 Accuracy: Authoritative Truth & Conflict Resolution
Accuracy assesses whether the grounding data represents verifiable, authorized factual truth.
Single Source of Truth (SSOT): Enterprise data estates frequently hold duplicate documents with conflicting data (e.g., an outdated 2024 Travel Policy PDF on a legacy SharePoint site versus the official 2026 Expense Policy in Dataverse). Agents will retrieve both unless authoritative sources are explicitly designated.
Precedence Hierarchies: When grounding spans multiple stores, architects must establish deterministic precedence rules: Core ERP (Dynamics 365 F&O) ≻ Operational CRM (Dataverse) ≻ Published Intranet (SharePoint) ≻ Informal Teams Chats \text{Core ERP (Dynamics 365 F\&O)} \succ \text{Operational CRM (Dataverse)} \succ \text{Published Intranet (SharePoint)} \succ \text{Informal Teams Chats} Core ERP (Dynamics 365 F&O) ≻ Operational CRM (Dataverse) ≻ Published Intranet (SharePoint) ≻ Informal Teams Chats
Source Verification & Provenance: Every retrieved grounding chunk must carry cryptographically verifiable or system-generated metadata ( source_uri , document_version , hash_id , last_modified_by ). Agents must cite these source attributes in final responses to enable human verification.
1.2 Relevance: Semantic Precision & Token Budget Optimization
Relevance measures how closely the retrieved grounding context addresses the user's specific business query without polluting the context window with distracting information.
The "Lost in the Middle" Phenomenon: Large Language Models exhibit degraded reasoning accuracy when key factual data is buried inside thousands of tokens of irrelevant background text. Pumping 50,000 tokens of raw policy documentation into context degrades reasoning performance.
Pre-Filtering vs. Post-Filtering:
Pre-Filtering (OData Filters in Azure AI Search): Filters the document index based on metadata attributes (e.g., tenant_id eq 'US' , department eq 'Finance' , security_clearance ge 3 ) before executing vector similarity calculations. This reduces the search space and guarantees tenant isolation.
Post-Filtering (Semantic Reranking): Evaluates the top K K K candidate chunks returned by dense vector search using a cross-encoder model to score semantic relevance against the exact query intent.
Context Pruning: Extracting only the relevant sub-paragraphs or tables from a 100-page document before injecting them into the agent's prompt context.
1.3 Timeliness: Freshness Latency, Caching & Real-Time Connectors
Timeliness evaluates the latency between an enterprise data update in the system of record and its availability to the agent's reasoning engine.
Architecture Freshness Latency Query Latency Compute Cost Ideal Use Case
Batch Vector Indexing Hours to Days Very Low (<200ms) Low (Scheduled batch jobs) Static HR policies, product manuals, archival records
Event-Driven Delta Indexing Near Real-Time (1-5 min) Very Low (<200ms) Moderate (Event Grid triggers Azure Functions) Customer support knowledge base updates, price book changes
Live Dynamic API Connector Instant (Zero lag) Moderate (500ms - 3s) Higher (Per-call API & connector overhead) Live inventory balances, real-time stock quotes, credit checks
Time-to-Live (TTL) & Cache Invalidation: When caching grounding responses or embeddings, architects must configure explicit TTL policies. When a master document in SharePoint or Dataverse is updated, a webhook or Azure Event Grid event must trigger immediate cache eviction.
1.4 Cleanliness: Normalization, Noise Removal & Structural Chunking
Cleanliness evaluates how free the source data is from navigational artifacts, syntactic garbage, and formatting anomalies that impair vector embedding quality.
Boilerplate & Noise Stripping: Raw enterprise files contain headers, footers, copyright notices, cookie disclaimers, and HTML CSS/script tags. If indexed, these repeated boilerplates artificially inflate vector similarity scores across unrelated queries.
Markdown Normalization: Transforming raw HTML, Word documents, and PDFs into clean, structured Markdown. Markdown preserves structural semantics—such as heading hierarchies ( # , ## , ### ), bulleted lists, and tables ( | Col1 | Col2 | )—which language models understand natively.
Structural & Semantic Chunking:
Fixed-size chunking (e.g., 500 characters with 50-character overlap): Fractures tables, splits sentences across chunks, and severs contextual links between headers and body paragraphs.
Layout-Aware / Hierarchical Chunking (Azure AI Document Intelligence): Segments documents along natural semantic boundaries (sections, headings, tables). Utilizes Parent-Child Chunking , where small child chunks (e.g., 200 tokens) are used for vector similarity matching, but the surrounding parent section (e.g., 1,000 tokens) is passed to the LLM for grounded context.
1.5 Availability: Uptime SLAs, Rate Limiting & Enterprise Isolation
Availability measures whether the grounding service can reliably meet runtime concurrency demands and latency SLAs without dropping requests.
Throughput & Throttling Limits: Azure OpenAI models enforce strict limits on Tokens Per Minute (TPM) and Requests Per Minute (RPM) . Azure AI Search enforces Queries Per Second (QPS) limits per search unit. If 5,000 enterprise users concurrently query an agent, unmitigated spikes trigger HTTP 429 (Too Many Requests) errors.
Resiliency Patterns: Implement Provisioned Throughput Units (PTU) for mission-critical production workloads to eliminate noisy-neighbor latency spikes, paired with exponential backoff retry policies in Azure API Management.
Network Security & Isolation: Enterprise grounding data must never transit the public internet. Architects must enforce Azure Private Endpoints , Virtual Network (VNet) integration, and Managed Identities, ensuring that Copilot Studio and Azure AI Foundry communicate with vector stores across private IP backbones.
2. Quantitative Scoring and Auditing Data Readiness
Prior to deploying an agent into production, solution architects must audit grounding data using objective quality benchmarks. The industry standard framework tested on the AB-100 is the RAG Triad : [ User Query ]
- / \
- Context Relevance / \ Answer Relevance
- v v
- [ Retrieved Context ] ----> [ Agent Response ]
- Groundedness (Faithfulness)
- The RAG Triad Metric Definitions
Context Relevance: Evaluates whether the retrieved grounding chunks are pertinent and focused on the user query, with minimal irrelevant noise. Context Relevance = Number of Relevant Sentences in Context Total Sentences in Retrieved Context \text{Context Relevance} = \frac{\text{Number of Relevant Sentences in Context}}{\text{Total Sentences in Retrieved Context}} Context Relevance = Total Sentences in Retrieved Context Number of Relevant Sentences in Context
Groundedness (Faithfulness): Measures whether every factual claim in the agent's generated response can be directly inferred from the retrieved grounding context. A response containing claims absent from the context represents a hallucination. Groundedness = Number of Output Claims Directly Supported by Grounding Context Total Factual Claims Made in Output \text{Groundedness} = \frac{\text{Number of Output Claims Directly Supported by Grounding Context}}{\text{Total Factual Claims Made in Output}} Groundedness = Total Factual Claims Made in Output Number of Output Claims Directly Supported by Grounding Context
Answer Relevance: Evaluates whether the agent's generated response directly addresses the user's initial question, regardless of whether the context was complete.
Data Readiness Audit Checklist
Readiness Pillar Audit Metric / Test Production Acceptance Target Remediation Action on Failure
Accuracy Conflicting source documents across indexed repositories 0 conflicting records Consolidate to Master Data Management (MDM) authoritative store
Relevance Context Relevance score on test evaluation dataset ≥ 0.85 \ge 0.85 ≥ 0.85 (85%) Refine chunking boundaries, implement metadata pre-filtering
Groundedness Faithfulness score on representative golden evaluation set ≥ 0.95 \ge 0.95 ≥ 0.95 (95%) Restrict system prompt temperature to 0.0, strengthen citation rules
Timeliness Maximum delta sync delay between update and searchability < 5 < 5 < 5 minutes for operational data Transition from batch indexing to Event Grid push synchronization
Cleanliness Percentage of chunks containing HTML tags or boilerplate < 1 % < 1\% < 1% of total chunks Update Document Intelligence extraction & noise-cleaning pipelines
Availability Grounding query latency at p95 under peak load < 350 ms < 350\text{ms} < 350 ms Scale Azure AI Search replicas, configure Redis caching layer
- 3. Real-World Architectural Case Scenario: Stale Grounding Data Remediation
- The Incident
During an annual Open Enrollment period, an enterprise HR Benefits Agent deployed in Microsoft Teams provided contradictory and obsolete healthcare deductibles to over 3,000 employees. Employees were informed that the in-network deductible was $500 (the 2025 rate), whereas the ratified 2026 plan required $750. Multiple employees made binding medical elections based on the agent's erroneous output, creating significant union grievance exposure.
Root Cause Analysis (RCA)
Vector Index Contamination: The Azure AI Search index contained both the legacy 2025_Benefits_Handbook.pdf (300 pages) and the newly uploaded 2026_Benefits_Addendum.pdf (12 pages).
Semantic Similarity Distortion: Because the 300-page 2025 handbook contained dozens of mentions of deductible terms with extensive supporting narrative, standard cosine vector search scored the 2025 document chunks higher than the concise 2-page table in the 2026 addendum.
Absence of Temporal Filtering: The search query did not include any date metadata constraints, allowing historical documents to compete equally with current policies.
Batch Indexing Lag: The index synchronization pipeline operated on a weekly cron job; several recent policy amendment memos uploaded to SharePoint 48 hours prior had not yet been processed into vectors. [User: What is my 2026 deductible?]
- |
- v
- [Standard Vector Search]
- / \
- v v
- [2025 Handbook Chunk] [2026 Addendum Chunk]
Cosine Score: 0.89 Cosine Score: 0.81 <-- Outdated 2025 chunk wins!
- |
- v
- [Agent Generates Erroneous $500 Response]
- The Architectural Remediation Pattern
To permanently resolve this vulnerability, the solution architect implements four architectural fixes:
Index Schema Enrichment with Temporal Metadata: Update the index schema to include mandatory OData filterable fields:
- effectiveStartDate (Edm.DateTimeOffset)
- effectiveEndDate (Edm.DateTimeOffset)
- isSuperseded (Edm.Boolean)
- documentVersion (Edm.String)
Mandatory OData Pre-Filtering: Configure the agent's knowledge retrieval query to execute an automated pre-filter enforcing active validity: isSuperseded eq false and effectiveStartDate le now() and (effectiveEndDate ge now() or effectiveEndDate eq null)
Freshness Scoring Profile with Date-Decay Boosting: Configure an Azure AI Search scoring profile that applies a mathematical decay function to document scores based on their lastModifiedDate :
Documents modified within the past 30 days receive an automatic score boost, ensuring newly ratified addenda outrank older base manuals.
Event-Driven Synchronization Pipeline: Replace the weekly batch crawler with an Azure Event Grid subscription on the SharePoint document library. When a new benefits document is uploaded or marked as approved, an event immediately triggers an Azure Function to extract, chunk, embed, and upsert the document within 90 seconds.