Study guide

6.2 Chat Completions API and Prompt Engineering

Azure AI Engineer Associate AI-102

Track your progress as you read

Finished sections get a check mark.

1 Introduction

About the Azure AI Engineer Associate (AI-102) Exam Study Strategies and Exam Tips for the AI-102

2 Domain 1: Plan and Manage an Azure AI Solution (20-25%)

1.1 Azure AI Services Overview and Selection 1.2 Resource Provisioning and Configuration 1.3 Security and Access Management 1.4 Responsible AI Principles and Implementation 1.5 Azure AI Solution Architecture Patterns

3 Content Safety and Moderation (within Plan and Manage, Domain 1)

2.1 Azure AI Content Safety Overview 2.2 Prompt Shields and Adversarial Attack Detection 2.3 Text and Image Moderation Implementation 2.4 Content Filtering in Azure OpenAI Service

4 Domain 4: Implement Computer Vision Solutions (10-15%)

3.1 Azure AI Vision — Image Analysis 4.0 3.2 Azure AI Face Service 3.3 Azure AI Custom Vision 3.4 Optical Character Recognition (OCR) 3.5 Video Analysis and the Retired Spatial Analysis

5 Domain 5: Implement Natural Language Processing Solutions (15-20%)

4.1 Azure AI Language — Text Analytics Features 4.2 Conversational Language Understanding (CLU) 4.3 Custom Text Classification and Custom NER 4.4 Azure AI Speech Services 4.5 Azure AI Translator Service 4.6 Question Answering and Conversational AI

6 Domain 6: Implement Knowledge Mining and Information Extraction Solutions (15-20%)

5.1 Azure AI Search — Fundamentals and Architecture 5.2 AI Enrichment with Skillsets 5.3 Azure AI Document Intelligence

7 Domain 2: Implement Generative AI Solutions (15-20%)

6.1 Azure OpenAI Service — Models and Deployment 6.2 Chat Completions API and Prompt Engineering 6.3 Embeddings and Vector Search for RAG 6.4 Fine-Tuning and Model Customization 6.5 Image Generation (gpt-image; DALL-E 3 Retired) 6.6 RAG After On Your Data, and AI Agents

8 Domain 3: Implement an Agentic Solution (5-10%)

3.1 Agentic Solutions: Foundry Agent Service and the Microsoft Agent Framework

9 Exam Review: Cross-Domain Topics and Advanced Practice

7.1 Service Comparison and Selection Guide 7.2 SDK and REST API Patterns 7.3 Responsible AI Across All Domains 7.4 End-to-End Solution Architectures 7.5 Exam Day Quick Reference

6.2 Chat Completions API and Prompt Engineering

Key Takeaways Explain This Topic AI

The Chat Completions API takes a messages array with three roles: system (behavior and guardrails), user (input), and assistant (prior model turns or few-shot examples).

On Azure the model parameter is the DEPLOYMENT name, not the model name — a top recurring AI-102 distractor.

Sampling is controlled by temperature (0-2) OR top_p (0-1); Microsoft says tune one, not both. Use max_tokens, frequency_penalty, presence_penalty, and stop to shape output.

Prompt-engineering patterns the exam tests: zero-shot, few-shot (user/assistant example pairs), chain-of-thought, and strict output-format specification.

Streaming (stream=True) returns tokens as deltas to cut perceived latency; tool/function calling lets the model request a structured call your code executes.

Last updated: October 2026

Quick Answer: Chat Completions uses system (behavior/guardrails), user (input), and assistant (prior turns or few-shot examples) messages. Steer output with temperature (creativity) or top_p (diversity) — never both — plus max_tokens , frequency_penalty , presence_penalty , and stop . On Azure the model argument is the deployment name .

Minimal Call from openai import AzureOpenAI

  • client = AzureOpenAI(
  • api_version="2024-10-21",
  • azure_endpoint="https://my-openai.openai.azure.com/",
  • azure_ad_token_provider=token_provider # managed identity preferred
  • )
  • resp = client.chat.completions.create(
  • model="gpt4o-chat", # DEPLOYMENT name, not "gpt-4o"
  • messages=[
  • {"role": "system", "content": "You are a concise Azure expert."},
  • {"role": "user", "content": "What is Azure AI Search?"}
  • ],
  • temperature=0.7,
  • max_tokens=500,
  • stop=None
  • )
  • print(resp.choices[0].message.content)
  • print(resp.usage.total_tokens)

On the Exam: With direct OpenAI you pass model="gpt-4o" ; with Azure OpenAI you pass the deployment name you chose . Expect at least one item where the wrong option is the literal model name.

Message Roles

  • Role Purpose Example
  • system Persona, scope, format, guardrails "You are a triage bot. Only answer health questions."
  • user Human input / data "What are flu symptoms?"
  • assistant Prior responses or few-shot answers "Fever, cough, body aches."

Multi-turn chat is stateless : you resend the full history every call because the API keeps no memory between requests. Long conversations therefore inflate input-token cost and can overflow the context window — a common scenario asking why costs rise turn over turn.

Sampling and Penalty Parameters

  • Parameter Range Default Effect
  • temperature 0-2 1 Higher = more random/creative; 0 = near-deterministic
  • top_p 0-1 1 Nucleus sampling: keep tokens within cumulative top_p
  • max_tokens 1-limit model-set Caps completion length
  • frequency_penalty -2 to 2 0 Positive reduces repeating tokens already used
  • presence_penalty -2 to 2 0 Positive pushes toward new topics
  • stop up to 4 strings none Halts generation at a sequence
  • n 1+ 1 Number of alternative completions
  • Choosing temperature
  • Goal Setting
  • Data extraction, code, factual Q&A temperature 0
  • Balanced helpfulness 0.3-0.5
  • Brainstorming, creative copy 0.7-1.0

On the Exam: Microsoft explicitly recommends adjusting temperature OR top_p, not both simultaneously. A question that sets temperature=0.2, top_p=0.2 together is testing this rule — the correct guidance is to modify only one.

Prompt-Engineering Patterns

Zero-shot — instruction only, no examples.

Few-shot — supply user/assistant example pairs so the model infers the pattern.

Chain-of-thought — "Think step by step" elicits intermediate reasoning, improving math/logic accuracy.

  • Output-format spec — demand strict JSON; pair with low temperature for reliability. messages = [
  • {"role": "system", "content": "Classify sentiment as positive, negative, or neutral."},
  • {"role": "user", "content": "The food was delicious!"},
  • {"role": "assistant", "content": "positive"},
  • {"role": "user", "content": "Service was terrible and slow."},
  • {"role": "assistant", "content": "negative"},
  • {"role": "user", "content": "It arrived on time and works perfectly!"}
  • ]

Trap: few-shot examples go in the assistant role, not stuffed into the system message — the exam distinguishes the two.

  • Streaming stream = client.chat.completions.create(
  • model="gpt4o-chat",
  • messages=[{"role": "user", "content": "Explain ML simply."}],
  • stream=True
  • )

for chunk in stream:

if chunk.choices and chunk.choices[0].delta.content:

print(chunk.choices[0].delta.content, end="", flush=True)

Streaming delivers tokens as deltas the moment they are produced, cutting perceived latency for chat UIs. It does not reduce total tokens or cost; the trade-off is that you cannot inspect the full response (or apply post-generation content checks) until the stream completes.

Tool / Function Calling

Function calling lets the model emit a structured request that your code executes — the model never runs the function itself. tools = [{

  • "type": "function",
  • "function": {
  • "name": "get_weather",
  • "description": "Get current weather for a city",
  • "parameters": {
  • "type": "object",
  • "properties": {
  • "location": {"type": "string"},
  • "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
  • },
  • "required": ["location"]
  • }
  • }
  • }]
  • resp = client.chat.completions.create(
  • model="gpt4o-chat",
  • messages=[{"role": "user", "content": "Weather in Seattle?"}],
  • tools=tools, tool_choice="auto"
  • )

if resp.choices[0].message.tool_calls:

  • call = resp.choices[0].message.tool_calls[0]
  • args = json.loads(call.function.arguments) # {"location": "Seattle, WA"}

The loop is: model returns tool_calls → your app runs the real function → you append a tool role message with the result → call the API again so the model writes the final natural-language answer.

Two behaviors matter for the exam. First, the model only proposes the call; it serializes the arguments as a JSON string you must parse, and your application is solely responsible for executing the work. Second, tool_choice governs the decision: auto lets the model choose whether to call a tool, none forbids tool use, and naming a function forces that specific call. You can register several tools at once and the model may request more than one in a single turn.

When to use which technique

  • Situation Reach for
  • Pull live data or trigger an action Tool / function calling
  • Enforce a structured task with no examples Zero-shot + format spec
  • Teach a pattern from samples Few-shot
  • Improve multi-step math or logic Chain-of-thought
  • Reduce perceived wait in a chat UI Streaming

On the Exam: Function calling is the foundation for the agent pattern in section 6.6 — the agent loop is just function calling plus managed conversations and memory. Expect items contrasting tool_choice values and asking who executes the function (always your code, never the service).