Study guide
3.1 Model Selection Strategy, Custom Models & Small Language Models (SLMs)
Microsoft Certified: Agentic AI Business Solutions Architect Expert AB-100
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Requirements Analysis & Grounding Data
1.1 Assessing Agents for Task Automation, Analytics & Decision-Making 1.2 Grounding Data Evaluation: Accuracy, Relevance, Timeliness, Cleanliness & Availability 1.3 Structuring Enterprise Business Data for AI System Interoperability
2 Chapter 2: AI Strategy, Cloud Adoption Framework & CoE
2.1 Cloud Adoption Framework (CAF) for Azure AI Strategy & Roadmap 2.2 Multi-Agent Architecture: Microsoft 365 Copilot, Copilot Studio & Foundry 2.3 Microsoft AI Center of Excellence (CoE) & Solution Governance 2.4 Developing Use Cases for Prebuilt Agents & the Microsoft Agent Catalog 2.5 Solution Rules & Constraints for Copilot Studio, Microsoft Foundry & Foundry Tools
3 Chapter 3: Model Selection, Prompt Engineering & Dynamics AI
3.1 Model Selection Strategy, Custom Models & Small Language Models (SLMs) 3.2 Enterprise Prompt Engineering Guidelines & Prompt Libraries 3.3 Cross-App AI Strategy for Multi-App Dynamics 365 Solutions
4 Chapter 4: ROI, TCO, Build vs Buy & Model Routing
4.1 ROI Criteria & Total Cost of Ownership (TCO) for AI Solutions 4.2 Comprehensive Financial ROI Analysis for Business Processes 4.3 Build vs Buy vs Extend Architectural Decision Framework 4.4 Dynamic Model Router Architecture & Cost-Performance Optimization
5 Chapter 5: Dynamics 365 AI & Contact Center Design
5.1 Business Terms & Copilot Customization in Customer Experience and Service 5.2 Connectors & Data Flow Design for Copilot in Dynamics 365 Sales 5.3 Agent Integration with Dynamics 365 Contact Center Channels
6 Chapter 6: Agent Archetypes & Copilot Studio Core
6.1 Designing Task Agents, Autonomous Agents & Prompt-and-Response Agents 6.2 Copilot Studio Topic Design, Trigger Phrases & Graceful Fallbacks 6.3 Orchestration Selection: Standard NLP vs CLU vs Generative AI Orchestration
7 Chapter 7: Copilot Studio Advanced Actions & Power Platform
7.1 Designing Agent Flows & Prompt Actions in Copilot Studio 7.2 Power Apps Canvas App Integration, Generative Pages & Agent Feeds 7.3 Power Platform Well-Architected Framework for AI Workloads 7.4 Proposing Microsoft Power Platform AI Features & the AI Hub
8 Chapter 8: Agent Extensibility, MCP & Computer Use
8.1 Microsoft Foundry Custom Models & Foundry Tools Selection 8.2 Agent Extensibility via Model Context Protocol (MCP) in Copilot Studio 8.3 Computer Use Automation & Web UI Interactions in Copilot Studio
9 Chapter 9: Advanced Behaviors & Prebuilt App Orchestration
9.1 Agent Behaviors: Multi-Step Reasoning, Voice Mode & M365 Agent Optimization 9.2 Orchestrating Prebuilt AI in Dynamics 365 F&O and M365 Copilot for Sales/Service 9.3 Orchestrating Prebuilt Copilot AI Features in Dynamics 365 Customer Experience & Service 9.4 Proposing Microsoft 365 Agents for Business Scenarios 9.5 Adding Knowledge Sources to In-App Help & Guidance in Dynamics 365 Finance & Supply Chain
10 Chapter 10: Telemetry, Monitoring & Agent Tuning
10.1 Monitoring Architectures: Azure Monitor, Application Insights & Studio Analytics 10.2 Telemetry Data Interpretation, Performance Metrics & Latency Tuning 10.3 User Feedback Analysis, Backlog Prioritization & Continuous Improvement
11 Chapter 11: Testing Strategy & Model Validation
11.1 Agent Testing Frameworks, Quality Metrics & Groundedness Benchmarks 11.2 Custom AI Model Validation, Safety Criteria & Regression Testing 11.3 Copilot-Assisted Test Case Generation & Cross-Dynamics End-to-End Testing 11.4 Validating Effective Copilot Prompt Practices
12 Chapter 12: ALM for Grounding Data & Copilot Studio Agents
12.1 ALM for Grounding Data, Vector Indexes & Knowledge Synchronization 12.2 ALM for Copilot Studio Agents, Connectors & Solution Packaging 12.3 Environment Strategy & Automated Deployment Pipelines for Studio Agents
13 Chapter 13: ALM for Foundry Agents & Dynamics 365 AI
13.1 ALM for Microsoft Foundry Agents Service & Model Registries 13.2 ALM for Custom AI Models, Fine-Tuning Artifacts & Deployment Slots 13.3 ALM for Dynamics 365 AI in Finance, Supply Chain & Customer Engagement
14 Chapter 14: Agent Identity, Security & Access Controls
14.1 Agent Identity Architecture, Microsoft Entra ID & Authentication Flows 14.2 Access Controls, Role-Based Security & Permissions on Grounding Data 14.3 Secure Data Processing & Grounding Data Isolation
15 Chapter 15: Model Security & Threat Mitigation
15.1 Model Security Architecture & Defense Against Vulnerabilities 15.2 Mitigating Prompt Manipulation, Jailbreaks & Indirect Prompt Injections 15.3 Azure AI Content Safety, Prompt Shields & Guardrails
16 Chapter 16: Responsible AI, Governance, Residency & Audit
16.1 Microsoft Responsible AI Standard: Principles, Assessment & Implementation 16.2 Data Residency, Sovereign Boundaries & Cross-Border Compliance 16.3 Comprehensive Audit Trails, Model Decision Logging & Microsoft Purview 16.4 Designing Governance for Agents: Environment Strategy, Data Policies & Agent Inventory
3.1 Model Selection Strategy, Custom Models & Small Language Models (SLMs)
Key Takeaways Explain This Topic AI
Frontier LLMs (GPT-4o, GPT-4o-mini) provide broad general knowledge, multimodal synthesis, and multi-step reasoning, whereas Small Language Models (Phi-3.5, Phi-4) deliver deterministic execution on focused tasks with ultra-low latency and minimal compute cost.
Latency-critical workloads requiring sub-50ms Time-to-First-Token (TTFT) or edge deployment on restricted hardware (using ONNX Runtime, DirectML, or Windows Copilot Runtime) are primary targets for SLMs such as Phi-3.5-mini and Phi-4.
Provisioned Throughput Units (PTUs) allocate dedicated model processing capacity with guaranteed latency SLAs and zero HTTP 429 throttling, contrasting with Pay-As-You-Go serverless consumption.
Supervised Fine-Tuning alters model weights to specialize in domain-specific vocabulary, strict syntax, or tone, whereas Retrieval-Augmented Generation (RAG) grounds models in dynamic enterprise data while preserving access control and factual provenance.
The Azure AI Foundry customization hierarchy dictates starting with Prompt Engineering, progressing to RAG for dynamic knowledge retrieval, and reserving Fine-Tuning for specialized formatting, latency optimization, or token-reduction needs.
Last updated: September 2026
3.1 Model Selection Strategy, Custom Models & Small Language Models (SLMs)
Quick Architecture Summary: Enterprise agentic solutions require a disciplined multi-model architecture. Frontier Large Language Models (LLMs) like GPT-4o excel at open-ended reasoning, multi-step orchestration, and multimodal synthesis, while Small Language Models (SLMs) such as Microsoft Phi-3.5 and Phi-4 offer high throughput, ultra-low latency, and cost-effective execution for deterministic classification, entity extraction, and edge computing. Fine-tuning must be reserved for domain vocabulary, strict formatting, and latency optimization—never as a substitute for dynamic grounding, which is strictly governed by Retrieval-Augmented Generation (RAG).
1. Model Selection Taxonomy: Frontier LLMs vs. Small Language Models (SLMs)
Architecting enterprise AI solutions on the Microsoft Cloud requires moving beyond a "one-size-fits-all" model strategy. In Azure AI Foundry and Microsoft Copilot Studio , architects can deploy a diverse spectrum of models spanning massive frontier foundation models down to compact, specialized Small Language Models (SLMs). +-----------------------------------------------------------------------------------------+
- | AZURE AI FOUNDRY MODEL CATALOG |
- +-----------------------------------------------------------------------------------------+
- | FRONTIER LLMs COMPACT LLMs SMALL LANGUAGE MODELS (SLMs) |
- | (GPT-4o) (GPT-4o-mini) (Phi-3.5-mini, Phi-4) |
- | - Multi-hop reasoning - High-speed reasoning - Sub-20ms TTFT |
- | - Broad world knowledge - Low-cost orchestration - Strict classification |
- | - Native multimodal - Function calling - JSON entity extraction |
- | - Complex coding & tools - RAG document synthesis - Edge / ONNX / NPU runtime |
- +-----------------------------------------------------------------------------------------+
- Frontier Large Language Models (LLMs)
GPT-4o ("omni"): Microsoft and OpenAI's flagship multimodal model. It accepts any combination of text, audio, image, and video inputs and generates text, audio, and image outputs. In agentic solutions, GPT-4o serves as the tier-one cognitive orchestrator —managing multi-agent coordination, resolving ambiguous user intent, handling multi-hop reasoning across unstructured data, and writing complex code or API transformations on the fly.
GPT-4o-mini: A high-speed, cost-efficient compact frontier model. It delivers near-frontier reasoning on standard tasks, excels at structured JSON extraction, and supports function calling at a fraction of the token cost and latency of GPT-4o. It represents the standard default model for high-frequency Copilot Studio orchestrations, summarization steps, and lightweight agent actions.
Small Language Models (SLMs): The Microsoft Phi Family
Microsoft's Phi family represents a paradigm shift in parameter efficiency. Rather than scraping raw web data, Phi models are trained on highly curated synthetic datasets, textbook-quality educational data, and filtered high-reasoning web corpuses ("Textbooks Are All You Need").
Phi-3.5-mini (3.8B parameters): A lightweight dense model supporting a 128K token context window. It matches or exceeds the benchmark performance of many models twice its size on reasoning, math, and code generation. It is designed specifically for constrained environments, low-latency API endpoints, and direct on-device execution.
Phi-3.5-MoE (16x3.8B parameters / 6.6B active parameters): A Mixture-of-Experts architecture that dynamically routes tokens to specialized expert sub-networks. It delivers the reasoning breadth of a 14B+ model while maintaining the inference compute profile and token speed of a 6.6B model.
Phi-4 (14B parameters): A state-of-the-art 14-billion parameter model optimized for complex mathematical reasoning, multi-turn synthetic instruction following, and dense structured tasks. Phi-4 bridges the gap between traditional SLMs and massive frontier models, frequently matching GPT-4-class performance on formal logic, technical analysis, and complex code tasks.
2. Architectural Trade-Off Dimensions: Reasoning, Latency, Cost & Topologies
When selecting a model for a specific agentic action or Copilot topic, the business solutions architect must evaluate four primary dimensions: [REASONING DEPTH]
- (GPT-4o, Phi-4)
- / \
- / \
- / \
- / \
- / \
- [LOW LATENCY] ------- [LOW COST / EDGE]
- (Phi-3.5-mini) (GPT-4o-mini, Local ONNX)
- 1. Reasoning Capability vs. Parameter Footprint
Frontier models possess massive parameter counts that enable deep semantic world knowledge, creative linguistic synthesis, and zero-shot generalization across disparate domains. However, for specialized business workflows—such as classifying an incoming field service incident or validating purchase order line items against tax tables—frontier open-ended reasoning is unnecessary. SLMs trained on dense reasoning corpora excel at executing targeted, deterministic tasks without wandering into speculative or verbose generations.
2. Latency Profiles & Throughput
In conversational agentic systems, latency directly governs user adoption. Two critical latency metrics must be optimized:
Time-to-First-Token (TTFT): The duration between the user prompt arriving at the endpoint and the generation of the first token. Frontier models typically exhibit TTFT between 250ms and 800ms depending on prompt length and server queuing. Compact SLMs like Phi-3.5-mini can achieve TTFT below 25ms on dedicated or edge compute.
Tokens Per Second (TPS) / Generation Throughput: The sustained speed of generation. High-throughput back-office batch agents (e.g., parsing 500,000 invoices overnight) require maximum TPS to avoid processing backlogs. SLMs deliver significantly higher generation throughput per dollar of compute.
3. Cost & Consumption Economics
Azure AI Foundry provides multiple billing models for model consumption:
Pay-As-You-Go (PAYG) Serverless Consumption: Billed per 1,000 prompt tokens and completion tokens. Ideal for unpredictable, sporadic, or low-to-medium volume traffic. However, during enterprise peak surges, PAYG endpoints are subject to shared regional Token-Per-Minute (TPM) quota limits and potential HTTP 429 throttling.
Provisioned Throughput Units (PTUs): Dedicated compute reservations providing guaranteed throughput and consistent latency SLAs. PTUs eliminate noisy-neighbor contention and rate limiting. PTUs are cost-effective when utilization is consistently above 60-70% of provisioned capacity, making them essential for mission-critical contact center bots and production customer-facing agents.
Managed Compute Endpoints (Virtual Machines): When hosting open-weights models like Phi-3.5 or fine-tuned custom models, organizations can deploy directly to Azure Machine Learning managed online endpoints backed by dedicated GPU VMs (e.g., NCv3 or NDv4 series), paying flat infrastructure rates.
4. Deployment Topologies: Cloud vs. Edge & On-Premises
While frontier models are exclusively available as cloud-hosted APIs in Azure AI Foundry, SLMs provide architectural flexibility across deployment topologies:
Cloud-Hosted Serverless API: Zero infrastructure management, automatic scaling, and consumption billing in Azure AI Foundry.
Edge and On-Premises Deployment: Organizations with strict data sovereignty, air-gapped environments, or intermittent connectivity (such as offshore drilling platforms, mining facilities, or factory floors) can deploy Phi-3.5 models locally using ONNX Runtime and DirectML . On Windows 11 enterprise devices, the Windows Copilot Runtime allows applications to execute Phi models directly on hardware Neural Processing Units (NPUs) with zero cloud network egress and zero per-token operational cost.
3. Comparative Matrix: Frontier LLMs vs. SLMs vs. Custom Models
Evaluation Metric Frontier LLM (GPT-4o) Compact LLM (GPT-4o-mini) Small Language Model (Phi-3.5-mini / Phi-4) Custom Fine-Tuned Model (Foundry)
Parameter Scale Hundreds of Billions Estimated ~8B - 15B 3.8B (Phi-3.5) / 14B (Phi-4) Base model dependent (3.8B to 8B)
Context Window 128,000 tokens 128,000 tokens 128,000 tokens Typically 16K to 128K tokens
Time to First Token (TTFT) Moderate (~300 - 800ms) Low (~150 - 300ms) Ultra-low (<25 - 60ms) Ultra-low to Low (<50 - 150ms)
Relative Token Cost High (Premium Tier) Very Low ($0.15/1M input) Lowest (Free on Local NPU) Flat VM rate or Dedicated PAYG
Multi-Hop Reasoning Exceptional High Moderate (Phi-3.5) / High (Phi-4) High on specialized target task
Edge / NPU Compatibility No (Cloud Only) No (Cloud Only) Yes (ONNX / DirectML / NPU) Yes (if fine-tuning an SLM base)
Optimal Enterprise Role Tier-1 cognitive orchestrator, multi-agent planner, complex synthesis Default agent worker, high-frequency RAG summarization, tool calling Edge processing, high-volume classification, deterministic extraction Proprietary jargon, non-standard schemas, strict latency optimization
4. Enterprise Use Cases for Small Language Models (SLMs)
Implementing an SLM is an architectural best practice when the business requirement demands high throughput, tight latency, and predictable execution. Core use cases include:
1. High-Throughput Ticket & Document Triage
In enterprise customer service organizations receiving hundreds of thousands of customer support tickets, emails, or chat inquiries daily, using GPT-4o for initial triage is economically wasteful. A Phi-3.5-mini model can parse the incoming text, classify the issue into one of 20 predefined enterprise categories, assign an initial urgency score, and route the ticket to the appropriate Dynamics 365 queue within 30 milliseconds.
2. Schema-Conformant Entity Extraction
Extracting structured entities—such as invoice line items, purchase order numbers, flight confirmation codes, or equipment serial numbers—from unstructured communications. SLMs fine-tuned or prompted with strict JSON schemas achieve parity with frontier models while processing transactions at a fraction of the cost.
3. Edge Execution in Disconnected & Regulated Scenarios
In Field Service scenarios where technicians operate in remote underground tunnels, maritime vessels, or secure defense facilities without internet access, SLMs running locally on ruggedized laptops via ONNX Runtime can analyze error logs, search local technical manuals, and suggest diagnostic troubleshooting steps without cloud connectivity.
4. Intelligent Router / Model Cascading Architecture
Rather than routing all user queries to a single model, architects implement a Model Router Pattern . A lightweight Phi-3.5 model acts as a front-door gatekeeper: it evaluates query complexity, determines whether the request is a simple informational lookup or a complex multi-step reasoning problem, and routes the query accordingly. Up to 70% of enterprise queries can be resolved by SLMs or compact models, reserving GPT-4o compute solely for complex edge cases.
5. The Customization Spectrum: Prompt Engineering vs. RAG vs. Fine-Tuning
A critical requirement for the AB-100 exam is knowing precisely when to configure Prompt Engineering, when to deploy Retrieval-Augmented Generation (RAG), and when to invest in Supervised Fine-Tuning. THE CUSTOMIZATION SPECTRUM
- +-----------------------+-----------------------------+-----------------------------+
- | PROMPT ENGINEERING | RAG | FINE-TUNING |
- +-----------------------+-----------------------------+-----------------------------+
- | - In-context learning | - Dynamic enterprise facts | - Internalized vocabulary |
- | - Zero training cost | - Security trimming (RBAC) | - Rigid non-standard syntax |
- | - Instant iteration | - Verifiable citations | - Reduced prompt token size |
- | - Zero infra overhead | - Real-time Dataverse sync | - SFT in Azure AI Foundry |
- +-----------------------+-----------------------------+-----------------------------+
- Prompt Engineering
Mechanisms: Modifying system messages, providing few-shot canonical examples, defining explicit delimiters, and structuring input templates.
When It Suffices: For 80% of enterprise use cases. When the base foundation model already possesses general linguistic and reasoning ability, and the required task can be clearly instructed within the context window.
Limitations: Consumes prompt tokens on every call; susceptible to context window limits; performance degrades if instructions become excessively convoluted.
Retrieval-Augmented Generation (RAG)
Mechanisms: Indexing enterprise documents in Azure AI Search or querying Microsoft Dataverse , retrieving semantically relevant text chunks, applying security trimming based on user identity (Entra ID), and injecting the retrieved context into the model's prompt.
When It Is Mandatory: Whenever the agent must answer questions grounded in dynamic, frequently changing enterprise data (e.g., current HR policies, internal knowledge base articles, live inventory balances). RAG provides verifiable citations, mitigates hallucinations, and enforces role-based access control.
Exam Trap Warning: Never attempt to use fine-tuning to update an AI model with new facts or current enterprise documents! Models do not reliably memorize factual knowledge through fine-tuning, and fine-tuning cannot enforce document-level user security trimming.
Fine-Tuning in Azure AI Foundry
Mechanisms: Supervised Fine-Tuning (SFT) involves training a pre-trained base model (e.g., GPT-4o-mini, Phi-3.5, or Babbage-002) on a curated dataset of hundreds or thousands of prompt-response pairs in JSONL format, altering the model's underlying weights.
When Fine-Tuning Is the Correct Architectural Decision:
Proprietary Domain Vocabulary & Jargon: When an enterprise uses highly specialized industry acronyms, technical terminology, or internal dialect that base models consistently misinterpret.
Strict Output Formatting & Non-Standard DSLs: When downstream legacy systems require an exact, non-standard domain-specific language (DSL) or esoteric XML/JSON serialization where zero syntax errors are tolerated.
Drastic Latency & Cost Optimization: When an organization wants to eliminate a lengthy 2,000-token system prompt containing 15 few-shot examples. By baking the instructions and style directly into the model's weights, the system prompt can be reduced to 50 tokens, slashing TTFT and token costs across millions of requests.
Performance Distillation: Fine-tuning a smaller, cheaper model (like GPT-4o-mini or Phi-3.5) on synthetic outputs generated by GPT-4o to achieve frontier-level accuracy on a narrow, repetitive task.
Azure AI Foundry Fine-Tuning Lifecycle
Data Preparation: Format training and validation datasets into the Chat Completion JSONL format containing system, user, and assistant message objects.
Validation Split: Reserve at least 10-20% of data for validation to monitor validation loss and prevent overfitting.
Hyperparameter Selection: Configure learning rate multiplier, batch size, and number of training epochs (typically 1 to 3 epochs to avoid catastrophic forgetting).
Model Evaluation: Evaluate model checkpoints against baseline metrics (perplexity, training loss, validation loss, and task-specific benchmark accuracy).
Deployment: Deploy the fine-tuned checkpoint to a dedicated managed online endpoint or provisioned deployment slot within Azure AI Foundry.
- 6. Real-World Architectural Case Scenario: High-Volume Claims Triage & Model Routing
- The Incident
A national insurance carrier launched an automated claims processing agent using a single GPT-4o deployment under Pay-As-You-Go serverless consumption. During severe weather events, daily claim volume spiked to 650,000 submissions. The architecture experienced catastrophic failures:
Widespread HTTP 429 Throttling: Regional Token-Per-Minute (TPM) quotas were rapidly exhausted, leaving incoming claims stuck in pending queues.
Runaway Operational Expenditure: Monthly token billing surged past $280,000 for simple entity extraction and policy categorization tasks.
Latency SLA Breaches: Average Time-to-First-Token exceeded 1,400ms due to multi-tenant queuing contention, violating the corporate 250ms processing SLA.
Root Cause Analysis (RCA)
The architecture team treated model selection as a homogeneous platform choice rather than a tiered capability routing problem. Over 85% of incoming claims followed standard automobile or property damage reporting formats requiring only categorical triage and policy number extraction—tasks requiring zero open-ended multi-hop reasoning. Furthermore, relying on shared serverless consumption without reserved capacity made the workload vulnerable to regional noisy neighbors.
The Architectural Remediation Pattern
The solution architect refactored the solution into a Cascading Model Router Architecture :
Tier-1 SLM Gatekeeper: Deployed a fine-tuned Phi-3.5-mini model on dedicated Azure Machine Learning managed online endpoints. The model handles document triage, urgency classification, and schema extraction in under 35ms at flat infrastructure cost, successfully resolving 82% of all submissions.
Dynamic Policy Grounding via RAG: Connected claims requiring policy verification to Azure AI Search using hybrid vector/keyword search with Entra ID security trimming, feeding retrieved clauses into GPT-4o-mini .
Frontier Escalation via PTUs: Reserved a baseline block of Provisioned Throughput Units (PTUs) for GPT-4o , dedicated strictly to the 18% of claims involving contested liability, potential fraud indicators, or multi-party litigation requiring deep multi-hop reasoning.
7. Architectural Decision Matrix
Technical Requirement Recommended Architecture Implementation Mechanism
Ground answers in weekly updated employee benefit manuals RAG Azure AI Search + Entra ID Security Trimming + GPT-4o-mini
Classify 1,000,000 IoT error logs per day with <30ms latency Fine-Tuned SLM Fine-tuned Phi-3.5-mini deployed to dedicated GPU container endpoint
Answer open-ended strategic questions across multiple corporate PDFs Frontier RAG Azure AI Search (Hybrid Search + Semantic Ranker) + GPT-4o
Convert messy customer email requests into proprietary ERP EDI syntax Fine-Tuned Model Fine-tuned GPT-4o-mini or Phi-4 trained on historical EDI pairs
Field technician assistant operating inside an underground mine without Wi-Fi Local SLM on Edge Phi-3.5-mini running on ruggedized Windows laptop via ONNX Runtime / NPU
Multi-agent planning, tool selection, and dynamic workflow synthesis Frontier LLM GPT-4o deployed with Provisioned Throughput Units (PTUs)
Tip
AB-100 Exam Tip: When an exam question describes high-volume, low-latency classification or data extraction, immediately look for an SLM (Phi-3.5 / Phi-4) or a fine-tuned compact model. When the scenario involves dynamically changing knowledge or strict user security permissions, RAG is always the correct choice. If the question asks how to eliminate rate-limiting errors (HTTP 429) during peak business hours, the answer is Provisioned Throughput Units (PTU).