Security, Networking, Monitoring, and Responsible AI

Microsoft Azure AI Apps and Agents Developer Associate AI-103

Track your progress as you read

Finished sections get a check mark.

1 AI-103 Blueprint, Microsoft Foundry, and Solution Planning

Current AI-103 Exam Facts Foundry Hubs, Projects, Models, and Deployments Security, Networking, Monitoring, and Responsible AI

2 Generative AI, Agents, and Retrieval-Augmented Generation

Prompt Design, Model Selection, and Structured Outputs RAG, Grounding, Embeddings, and Azure AI Search Agent Tools, Functions, Memory, and Evaluation

3 Vision, Language, Information Extraction, and Final Review

Vision, Multimodal, Speech, and Language Workloads Image and Video Generation, Editing, and Content Understanding Pipelines Document Intelligence, Content Understanding, and Extraction AI-103 Study Plan, Traps, and Practice Strategy

Security, Networking, Monitoring, and Responsible AI

Key Takeaways Explain This Topic AI

Production AI-103 solutions should prefer Microsoft Entra ID, managed identity, and RBAC over embedded API keys.

Private endpoints, private link, and managed networking reduce public exposure for Foundry projects, AI Search, Storage, APIs, and other dependencies.

Content filters address harmful content categories, while Prompt Shields target direct and indirect prompt-injection attacks.

Monitoring and tracing are separate but related: monitoring tracks production health and quality trends, while tracing captures request-level execution evidence.

Responsible AI planning includes evaluators, safety evaluations, provenance metadata, approval workflows, tool constraints, and user-feedback loops.

Last updated: June 2026

Security, Networking, Monitoring, and Responsible AI

AI-103 treats security and responsible AI as design requirements, not cleanup tasks. A Foundry app might pass a quick demo with an API key, public endpoints, and no telemetry, but that design will not satisfy production scenarios. The exam favors keyless authentication , scoped authorization, private connectivity, safety filters, observability, and human oversight where actions are sensitive.

Identity and Access

Use Microsoft Entra ID and managed identity when Azure-hosted compute calls Foundry Models, Azure OpenAI, Azure AI Search, Storage, Key Vault, or internal APIs. A managed identity is an Azure-managed service principal, so code can request tokens without storing secrets. Then use role-based access control (RBAC) to grant only the permissions required at the correct scope.

Requirement Better design Risky design

App calls model deployment Managed identity with Azure AI or Azure OpenAI data-plane role API key hard-coded in source or app settings with broad access

Project stores reusable service access Project connection using Entra ID or managed identity where possible Shared admin key pasted into multiple flows

Developer needs project access Azure AI User or project-scoped role Subscription Owner for day-to-day development

Secret still required Store in Key Vault and retrieve with managed identity Store in repository, prompt flow file, or front-end code

Networking and Isolation

Use private networking when data sensitivity, compliance, or exfiltration risk matters. Private endpoints and private link create private IP paths to services such as Foundry resources, Storage, Azure AI Search, API Management, or private tool endpoints. In stricter designs, disable public network access and make sure every dependency has a private route, not just the model endpoint.

For agents, also constrain tools. Register only the tools the agent needs, authenticate tool calls independently, and require approval before high-impact actions such as sending external messages, updating production records, executing code, or calling privileged APIs.

Safety Controls

Content safety has multiple layers. Azure AI Content Safety content filters classify both prompts and completions across four harm categories: hate, sexual, violence, and self-harm . Each category is scored on a severity scale, and Azure OpenAI exposes configurable thresholds of safe, low, medium, and high so you choose how aggressively to block. The default filter blocks medium and high severity; a regulated workload may tighten to block low and above, while an internal research tool may loosen it.

Prompt Shields look for direct jailbreak attempts and indirect prompt-injection attacks hidden in retrieved documents, web pages, emails, images, or uploaded files. Protected-material filters detect known copyrighted text or code in generated output, and groundedness detection flags ungrounded (fabricated) claims in RAG responses.

Control Main purpose Exam clue

Content filter (4 categories, 4 severities) Block or annotate unsafe input and output Harmful prompt or generated response

Prompt Shields Detect jailbreak and indirect prompt injection Retrieved content tries to override system instructions

Protected material detection Flag known copyrighted text or code Output reproduces lyrics, articles, or licensed code

Groundedness detection / evaluator Check whether an answer is supported by sources RAG bot invents facts or ignores citations

Approval workflow (human-in-the-loop) Keep humans in control of sensitive actions Agent can spend money, change data, or contact users

Note the difference between a real-time filter (blocks or annotates a single request) and an evaluator (scores quality offline or on sampled traffic). Both can check groundedness, but only the filter intervenes in-line; the evaluator measures and reports.

Monitoring, Tracing, and Evaluation

Monitoring answers: "Is production healthy?" Foundry monitoring integrates with Azure Monitor and Application Insights to surface token consumption, latency, exceptions, quality scores, safety events, and response trends. Use alerts when cost spikes, latency rises, groundedness drops, or harmful-output metrics breach thresholds.

Tracing answers: "What happened in this one request?" Distributed traces record spans for prompt construction, retrieval, model calls, tool invocations, agent decisions, and downstream services. Traces support debugging, audit review, and provenance because they show the evidence trail behind a response.

Evaluation answers: "Is this system good enough?" Run pre-production evaluation on labeled datasets and use continuous evaluation on sampled production traffic. Include quality metrics, risk and safety evaluators, groundedness, relevance, task completion, and custom business checks. A reliable AI-103 solution combines all three: monitoring for trends, tracing for detail, and evaluation for measurable quality.

Foundry ships built-in evaluators you should be able to match to a goal. Groundedness and relevance check RAG answer quality against retrieved context; coherence and fluency check readability; similarity and F1/BLEU/ROUGE compare output to ground-truth labels; retrieval evaluators score whether the right chunks were fetched; and risk and safety evaluators score for violence, hate, sexual, self-harm, and protected-material content. For agents specifically, evaluate intent resolution, task adherence, and tool-call accuracy .

Tracing in Foundry is built on OpenTelemetry semantic conventions for generative AI, so spans are portable to Application Insights and other backends.

Scenario: Diagnosing a Bad RAG Answer

Users report the support bot occasionally cites the wrong policy. The correct triage order uses all three pillars. Monitoring shows that groundedness scores dipped after a knowledge-base reindex (a trend). Tracing for one failing request shows the retrieval span returned the wrong document chunk before the model ever ran (the detail). Evaluation on a labeled set then confirms the fix: after tuning the index and hybrid-search settings, the retrieval and groundedness evaluators recover.

The exam-correct lesson is that a wrong answer is usually a retrieval problem, surfaced by monitoring, pinpointed by tracing, and proven fixed by evaluation, not a reason to swap to a bigger model.