Study guide
6.6 RAG After On Your Data, and AI Agents
Azure AI Engineer Associate AI-102
Track your progress as you read
Finished sections get a check mark.
1 Introduction
About the Azure AI Engineer Associate (AI-102) Exam Study Strategies and Exam Tips for the AI-102
2 Domain 1: Plan and Manage an Azure AI Solution (20-25%)
1.1 Azure AI Services Overview and Selection 1.2 Resource Provisioning and Configuration 1.3 Security and Access Management 1.4 Responsible AI Principles and Implementation 1.5 Azure AI Solution Architecture Patterns
3 Content Safety and Moderation (within Plan and Manage, Domain 1)
2.1 Azure AI Content Safety Overview 2.2 Prompt Shields and Adversarial Attack Detection 2.3 Text and Image Moderation Implementation 2.4 Content Filtering in Azure OpenAI Service
4 Domain 4: Implement Computer Vision Solutions (10-15%)
3.1 Azure AI Vision — Image Analysis 4.0 3.2 Azure AI Face Service 3.3 Azure AI Custom Vision 3.4 Optical Character Recognition (OCR) 3.5 Video Analysis and the Retired Spatial Analysis
5 Domain 5: Implement Natural Language Processing Solutions (15-20%)
4.1 Azure AI Language — Text Analytics Features 4.2 Conversational Language Understanding (CLU) 4.3 Custom Text Classification and Custom NER 4.4 Azure AI Speech Services 4.5 Azure AI Translator Service 4.6 Question Answering and Conversational AI
6 Domain 6: Implement Knowledge Mining and Information Extraction Solutions (15-20%)
5.1 Azure AI Search — Fundamentals and Architecture 5.2 AI Enrichment with Skillsets 5.3 Azure AI Document Intelligence
7 Domain 2: Implement Generative AI Solutions (15-20%)
6.1 Azure OpenAI Service — Models and Deployment 6.2 Chat Completions API and Prompt Engineering 6.3 Embeddings and Vector Search for RAG 6.4 Fine-Tuning and Model Customization 6.5 Image Generation (gpt-image; DALL-E 3 Retired) 6.6 RAG After On Your Data, and AI Agents
8 Domain 3: Implement an Agentic Solution (5-10%)
3.1 Agentic Solutions: Foundry Agent Service and the Microsoft Agent Framework
9 Exam Review: Cross-Domain Topics and Advanced Practice
7.1 Service Comparison and Selection Guide 7.2 SDK and REST API Patterns 7.3 Responsible AI Across All Domains 7.4 End-to-End Solution Architectures 7.5 Exam Day Quick Reference
6.6 RAG After On Your Data, and AI Agents
Key Takeaways Explain This Topic AI
Azure OpenAI On Your Data, which attached a data source to chat completions for managed RAG with citations, was retired on October 14, 2026 and is no longer supported.
For an Azure AI Search-backed app, Microsoft's migration path is a Foundry IQ knowledge base that reuses the existing index; the generally available alternatives are knowledge base retrieval (2026-04-01) or classic RAG with app-managed answer generation.
AI agents extend LLMs with planning plus tool calling, code interpreter, file search/retrieval, and persistent memory to complete multi-step tasks autonomously.
Microsoft Foundry Agent Service is the managed platform: create a versioned agent (model + instructions + tools) -> create a conversation -> create a response -> return a function_call_output for each function_call -> read output_text. The classic thread/run API is deprecated and retires on March 31, 2027.
Pick the simplest approach that works: classic RAG or a Foundry IQ knowledge base for document Q&A, custom RAG for full control, agents for dynamic multi-step tool-driven workflows.
Last updated: October 2026
Quick Answer: Azure OpenAI On Your Data (the data_sources extension that attached an index to chat completions) was retired on October 14, 2026 . For an Azure AI Search-backed app, Microsoft's migration path is a Foundry IQ knowledge base that reuses the existing index; classic RAG with app-managed answer generation is a generally available alternative. Agents add planning + tool calling to do multi-step work. Foundry Agent Service is the managed agent platform: create a versioned agent → conversation → response → return function_call_output items → read the result.
Azure OpenAI On Your Data (Retired)
The final AI-102 outline asks you to "implement a RAG pattern by grounding a model in your data". Until its retirement, Azure OpenAI On Your Data did this without retrieval code: a chat completions request carried a data_sources block (legacy dataSources , or the /extensions/chat/completions path) pointing at Azure AI Search, Blob Storage, uploaded files, URLs, Azure Cosmos DB for MongoDB, Elasticsearch, or MongoDB Atlas, and the service returned grounded answers with citations. Microsoft retired it on October 14, 2026 . Ordinary chat completions requests without that configuration are not affected.
Grounding Now: Foundry IQ or Classic RAG
Path What you build Status
Foundry IQ knowledge base Reuses your existing Azure AI Search index for retrieval, answers, and citations Microsoft's migration sample uses answer synthesis in 2026-08-01-preview
Knowledge base retrieval Minimal, extractive retrieval with 2026-04-01 ; your app generates the answer Generally available
Classic RAG (section 6.3) Your code queries Azure AI Search (hybrid + semantic ranker) and grounds the prompt Generally available
Foundry Agent Service + Foundry IQ An agent that uses the knowledge base plus other tools, actions, or workflows For more than corpus Q&A
Migration isn't automatic: you map fields and filters, enforce per-user authorization in your backend on every request, validate citation IDs, and keep conversation history in your application.
On the Exam: A 2026 design should not call On Your Data. For a standard "answer questions over my documents" requirement, ground the model through Azure AI Search with hybrid search and the semantic ranker (classic RAG) or a Foundry IQ knowledge base, and instruct the model in the system message to answer only from the retrieved content.
AI Agents (Agentic AI)
An agent wraps an LLM with the ability to plan, call tools, run code, retrieve documents, and remember state — turning a one-shot chat into an autonomous multi-step worker.
- Capability What it adds Example
- Tool / function calling Invoke your APIs Query a database, hit a weather API
- Code interpreter Write & execute code in a sandbox Analyze a CSV, make a chart
- File search Built-in RAG over uploaded files Cite a PDF spec
- Multi-step reasoning Decompose tasks Research → analyze → summarize
- Memory (conversations) Persist conversation state Recall earlier user constraints
- Microsoft Foundry Agent Service Lifecycle (azure-ai-projects 2.x) import json
- from azure.ai.projects import AIProjectClient
- from azure.ai.projects.models import PromptAgentDefinition, FunctionTool, CodeInterpreterTool
- from azure.identity import DefaultAzureCredential
- project = AIProjectClient(credential=DefaultAzureCredential(),
- endpoint="https://my-resource.services.ai.azure.com/api/projects/my-project")
- agent = project.agents.create_version( # 1. versioned agent
- agent_name="product-assistant",
- definition=PromptAgentDefinition(
- model="gpt-4o",
- instructions="Help customers find and compare products.",
- tools=[FunctionTool(name="search_products", parameters={...},
- description="Search the product catalog"),
- CodeInterpreterTool()]))
- openai = project.get_openai_client(agent_name=agent.name)
- conversation = openai.conversations.create() # 2. conversation (was: thread)
- response = openai.responses.create( # 3. response (was: run)
- conversation=conversation.id,
- input="Find a laptop under $1000 with 16GB RAM")
- while any(item.type == "function_call" for item in response.output): # 4. tool loop
- outputs = [{"type": "function_call_output", "call_id": item.call_id,
- "output": json.dumps(run_tool(item.name, json.loads(item.arguments)))}
- for item in response.output if item.type == "function_call"]
- response = openai.responses.create(conversation=conversation.id, input=outputs)
print(response.output_text) # 5. final answer
The function call is the heart of the pattern: when the agent decides to use one of your functions, the response contains a function_call item instead of a final answer. Your code runs the function and sends a function_call_output item with the same call_id in the next response, and this can repeat several times before the agent answers. The classic threads-and-runs API showed the same pause as a run in requires_action until you called submit_tool_outputs ; Microsoft has deprecated those classic agents and retires them on March 31, 2027.
Choosing a RAG/Agent Approach
- Approach Effort Control Best for
- Foundry IQ knowledge base Low Managed retrieval and citations Corpus Q&A over an existing Search index
- Custom RAG (code) Medium Full Custom ranking/pre/post-processing
- Agent + tools High Maximum Dynamic multi-step, tool-driven tasks
- Managed retrieval vs. building it yourself
A Foundry IQ knowledge base handles retrieval and citations for you; a hand-built pipeline (section 6.3) gives full control of chunking, ranking, filtering, and post-processing. Choose the managed path for standard "answer questions over my documents" requirements, and choose custom RAG when a scenario demands bespoke ranking, filtering, or transformation of retrieved content before it reaches the model.
Why agents differ from plain RAG
The distinction the exam draws is autonomy and steps . A RAG call retrieves once and answers once. An agent can decide to search, then call an API, then run code, then answer — looping through function calls as many times as the task needs, carrying state in its conversation. That power costs complexity: more tokens, harder debugging, and the need to secure every tool the agent can invoke. Built-in tool types — code_interpreter for sandboxed code and file_search for managed RAG over uploaded files — let you add common capabilities without writing your own functions.
On the Exam: The 2026 AI-102 leans into agents. Memorize the current lifecycle (create agent version → conversation → response → return function_call_output → read result) and recognize the classic one (agent → thread → message → run → requires_action → submit_tool_outputs ), where requires_action = waiting for your tool output , not an error or completion. Prefer the simplest option that satisfies the requirement — don't reach for an agent when plain RAG or a single function call suffices.