Study guide

4.4 Azure AI Speech Services

Azure AI Engineer Associate AI-102

Track your progress as you read

Finished sections get a check mark.

1 Introduction

About the Azure AI Engineer Associate (AI-102) Exam Study Strategies and Exam Tips for the AI-102

2 Domain 1: Plan and Manage an Azure AI Solution (20-25%)

1.1 Azure AI Services Overview and Selection 1.2 Resource Provisioning and Configuration 1.3 Security and Access Management 1.4 Responsible AI Principles and Implementation 1.5 Azure AI Solution Architecture Patterns

3 Content Safety and Moderation (within Plan and Manage, Domain 1)

2.1 Azure AI Content Safety Overview 2.2 Prompt Shields and Adversarial Attack Detection 2.3 Text and Image Moderation Implementation 2.4 Content Filtering in Azure OpenAI Service

4 Domain 4: Implement Computer Vision Solutions (10-15%)

3.1 Azure AI Vision — Image Analysis 4.0 3.2 Azure AI Face Service 3.3 Azure AI Custom Vision 3.4 Optical Character Recognition (OCR) 3.5 Video Analysis and the Retired Spatial Analysis

5 Domain 5: Implement Natural Language Processing Solutions (15-20%)

4.1 Azure AI Language — Text Analytics Features 4.2 Conversational Language Understanding (CLU) 4.3 Custom Text Classification and Custom NER 4.4 Azure AI Speech Services 4.5 Azure AI Translator Service 4.6 Question Answering and Conversational AI

6 Domain 6: Implement Knowledge Mining and Information Extraction Solutions (15-20%)

5.1 Azure AI Search — Fundamentals and Architecture 5.2 AI Enrichment with Skillsets 5.3 Azure AI Document Intelligence

7 Domain 2: Implement Generative AI Solutions (15-20%)

6.1 Azure OpenAI Service — Models and Deployment 6.2 Chat Completions API and Prompt Engineering 6.3 Embeddings and Vector Search for RAG 6.4 Fine-Tuning and Model Customization 6.5 Image Generation (gpt-image; DALL-E 3 Retired) 6.6 RAG After On Your Data, and AI Agents

8 Domain 3: Implement an Agentic Solution (5-10%)

3.1 Agentic Solutions: Foundry Agent Service and the Microsoft Agent Framework

9 Exam Review: Cross-Domain Topics and Advanced Practice

7.1 Service Comparison and Selection Guide 7.2 SDK and REST API Patterns 7.3 Responsible AI Across All Domains 7.4 End-to-End Solution Architectures 7.5 Exam Day Quick Reference

4.4 Azure AI Speech Services

Key Takeaways Explain This Topic AI

Azure AI Speech provides speech-to-text (real-time and batch), text-to-speech with neural voices, speech translation, and pronunciation assessment; Speaker Recognition was retired on September 30, 2025.

Real-time STT uses recognize_once() for a single utterance and start_continuous_recognition() with event handlers for streaming audio.

SSML controls TTS output: <prosody> for rate/pitch/volume, <break> for pauses, <say-as> for interpretation, mstts:express-as for speaking styles, and <phoneme> for exact pronunciation.

Custom Speech adapts the base model with language data, audio+transcripts, or pronunciation files; Custom Neural Voice requires Microsoft approval (limited access).

Speech translation does speech-to-text and speech-to-speech translation in one pipeline by setting a recognition language and adding target languages.

Last updated: October 2026

Quick Answer: Azure AI Speech offers speech-to-text (STT) , text-to-speech (TTS) with neural voices, speech translation (real-time speech-to-text and speech-to-speech), and pronunciation assessment (Speaker Recognition was retired on September 30, 2025). You integrate via the Speech SDK (Python, C#, JavaScript) using a SpeechConfig with a key and region — region is mandatory because Speech routes to regional endpoints.

Speech-to-Text Modes

  • Mode Method / mechanism Use case
  • Single utterance recognize_once() short command, push-to-talk
  • Continuous start_continuous_recognition() + events meetings, dictation, live captions
  • Batch REST job over Blob audio bulk recorded files, async

Continuous recognition fires recognizing (partial, interim) and recognized (final) events; you must call stop_continuous_recognition() to end it. Batch transcription submits a job over audio in Blob Storage and supports word-level timestamps and speaker diarization in the JSON output. import azure.cognitiveservices.speech as speechsdk

  • cfg = speechsdk.SpeechConfig(subscription="<key>", region="eastus")
  • cfg.speech_recognition_language = "en-US"
  • rec = speechsdk.SpeechRecognizer(speech_config=cfg,
  • audio_config=speechsdk.AudioConfig(use_default_microphone=True))
  • result = rec.recognize_once()

if result.reason == speechsdk.ResultReason.RecognizedSpeech:

print(result.text)

elif result.reason == speechsdk.ResultReason.Canceled:

  • print(result.cancellation_details.reason) # inspect for auth/quota errors
  • Text-to-Speech and SSML

TTS uses neural voices named <locale>-<Name>Neural , e.g. en-US-JennyNeural . Speech Synthesis Markup Language (SSML) gives fine control:

  • Element Controls Example
  • <voice> which voice name="en-US-JennyNeural"
  • <prosody> rate, pitch, volume rate="slow" pitch="+10%"
  • <break> pauses time="500ms" / strength="strong"
  • <say-as> interpret content interpret-as="telephone"
  • <mstts:express-as> speaking style style="cheerful"
  • <phoneme> exact pronunciation IPA or SAPI alphabet

<emphasis> stress level="strong" <speak version="1.0" xmlns:mstts="https://www.w3.org/2001/mstts" xml:lang="en-US">

  • <voice name="en-US-JennyNeural">
  • <mstts:express-as style="cheerful">Welcome!</mstts:express-as>
  • <break time="500ms"/>
  • <prosody rate="slow" pitch="+5%">Twenty percent off today.</prosody>
  • <say-as interpret-as="telephone">+15551234567</say-as>
  • </voice>
  • </speak>

On the Exam: Match the element to the need — adjust speed/pitch → <prosody> ; insert a pause → <break> ; read digits as a phone number/date → <say-as> ; sound happy/sad → <mstts:express-as> ; force a pronunciation → <phoneme> .

Speech Translation

Use SpeechTranslationConfig : set speech_recognition_language once and call add_target_language() per target. One pipeline yields recognized source text plus translations to several languages simultaneously — more efficient than chaining STT then Translator. tc = speechsdk.translation.SpeechTranslationConfig(subscription="<key>", region="eastus")

  • tc.speech_recognition_language = "en-US"
  • for lang in ("fr", "de", "es"): tc.add_target_language(lang)
  • Pronunciation Assessment and the Retired Speaker Recognition
  • Capability What it does
  • Pronunciation assessment scores accuracy, fluency, completeness, prosody for language learning
  • Speaker diarization separates who spoke when in a transcript, with generic speaker labels

Speaker Recognition (1:1 speaker verification and 1:N speaker identification) was retired on September 30, 2025, so apps can no longer use it.

Custom Speech vs. Custom Neural Voice

Custom Speech improves recognition accuracy for jargon, product names, accents, or noisy audio. Training data types: language data (text for vocabulary), audio + transcripts (acoustic adaptation), and pronunciation files. Custom Neural Voice (CNV) creates a unique synthetic voice from your recordings and is limited access — it requires a Microsoft application and approval to prevent misuse.

Common Trap: "Improve transcription of medical terms" = Custom Speech , not Custom Neural Voice (which is for building a branded TTS voice). And whenever a scenario says "convert spoken English into written French and German together," choose Speech Translation , not separate STT plus Translator calls.

Authentication, Regions, and the Audio Pipeline

Every Speech SDK scenario starts with a SpeechConfig, and the exam expects you to know that it takes both a key and a region, or alternatively an authorization token for short-lived credentials. Unlike the Language and Translator services, the Speech endpoint is region-bound, so a config built with the wrong region simply cannot reach your resource. When a recognition result returns the Canceled reason, the right debugging move is to read cancellation_details, whose error code distinguishes an authentication failure from quota throttling or a bad audio format.

Audio input is configured separately from the recognizer through an AudioConfig. The three input paths are the default microphone, a WAV file by filename, and a push or pull stream for custom sources such as telephony or a browser. The base recognizer expects 16 kHz or 8 kHz, 16-bit, mono PCM WAV; feeding it stereo MP3 is a classic cause of NoMatch results. For very long or high-volume audio, batch transcription is preferred over continuous recognition because it runs server-side over files already in Blob Storage and can emit diarization and timestamps without holding an open connection.

Mapping Speech Capabilities to Requirements

Under time pressure, map the verb in the question to the service. "Transcribe" or "caption" points to speech-to-text; "read aloud" or "voice response" points to text-to-speech; "speak in our brand's unique voice" is the limited-access Custom Neural Voice; "recognize who is speaking" used to mean Speaker Recognition, which was retired on September 30, 2025 (diarization only separates speakers); and "score how well a learner pronounces words" is pronunciation assessment. When the requirement combines listening and translating, speech translation collapses the pipeline into one call.

Memorizing these verb-to-feature mappings, plus the SSML element table, covers the overwhelming majority of Speech questions on the exam. Remember too that speech translation can emit synthesized audio in the target language, not only text, when you set a synthesis voice, which is the distinction between speech-to-text translation and speech-to-speech translation that the exam sometimes tests directly.