Study guide

16.1 Advanced Fine-Tuning Methods (LoRA, QLoRA, Full)

Microsoft AI-300: Operationalizing Machine Learning and Generative AI Solutions Exam

Track your progress as you read

Finished sections get a check mark.

1 Introduction & Exam Overview

1.1 Current Exam Facts 1.2 Skills Measured, Format, and How This Guide Is Organized 1.3 Candidate Profile, Azure ML v2, Foundry, and DevOps Tooling

2 Chapter 2: Azure Machine Learning Workspaces, Datastores, Compute, and IAM

2.1 Create and Manage Azure Machine Learning Workspaces 2.2 Create and Manage Datastores 2.3 Create and Manage Compute Targets 2.4 Configure Identity and Access Management for Workspaces

3 Chapter 3: Azure Machine Learning Data Assets, Environments, Components, and Registries

3.1 Create and Manage Data Assets 3.2 Create and Manage Environments 3.3 Create and Manage Components 3.4 Share Assets Across Workspaces with Registries

4 Chapter 4: Infrastructure as Code, GitHub, and Network Isolation for Azure ML

4.1 GitHub Integration and Git Source Control for Machine Learning Projects 4.2 Deploy Workspaces and Resources with Bicep and Azure CLI 4.3 Automate Resource Provisioning with GitHub Actions 4.4 Restrict Network Access to Machine Learning Workspaces

5 Chapter 5: MLflow Tracking, AutoML, Training Jobs, and Hyperparameter Sweeps

5.1 Configure Experiment Tracking with MLflow 5.2 Use Automated Machine Learning to Explore Optimal Models 5.3 Notebooks, Command Jobs, and Training Scripts 5.4 Automate Hyperparameter Tuning

6 Chapter 6: Distributed Training, Training Pipelines, and Job Comparison

6.1 Manage Distributed Training for Large and Deep Learning Models 6.2 Implement Training Pipelines 6.3 Compare Model Performance Across Jobs

7 Chapter 7: Model Registration, Responsible AI, and Lifecycle Management

7.1 Register MLflow Models and Package Feature Retrieval Specs 7.2 Evaluate Models with Responsible AI Principles 7.3 Manage Model Lifecycle and Archiving

8 Chapter 8: Online Endpoints, Batch Endpoints, Rollout, and Rollback

8.1 Deploy Models to Managed Online Endpoints 8.2 Deploy Models to Batch Endpoints 8.3 Test, Troubleshoot, Progressive Rollout, and Safe Rollback

9 Chapter 9: Data Drift, Production Metrics, Retraining, and Alerts

9.1 Detect and Analyze Data Drift 9.2 Monitor Production Metrics and Trigger Retraining or Alerts

10 Chapter 10: Microsoft Foundry Resources, IAM, Networking, and IaC

10.1 Create and Configure Foundry Resources and Project Environments 10.2 Managed Identities and RBAC for Foundry 10.3 Foundry Network Security and Private Networking 10.4 Deploy Foundry Infrastructure with Bicep and Azure CLI

11 Chapter 11: Deploying and Managing Foundation Models

11.1 Serverless API Endpoints and Managed Compute for Foundation Models 11.2 Select Foundation Models for Specific Use Cases 11.3 Model Versioning and Production Deployment Strategies 11.4 Provisioned Throughput Units for High-Volume Workloads

12 Chapter 12: Prompt Design, Variants, and Git Version Control

12.1 Design and Develop Prompts 12.2 Create Prompt Variants and Compare Performance 12.3 Version Control for Prompts with Git

13 Chapter 13: Evaluation Datasets, Quality Metrics, Safety, and Automated Workflows

13.1 Test Datasets and Data Mapping for Evaluation 13.2 Groundedness, Relevance, Coherence, and Fluency 13.3 Risk and Safety Evaluations 13.4 Built-in and Custom Automated Evaluation Workflows

14 Chapter 14: Continuous Monitoring, Performance, Cost, Tracing, and Logging

14.1 Continuous Monitoring in Microsoft Foundry 14.2 Latency, Throughput, Token Use, and Cost Metrics 14.3 Logging, Distributed Tracing, and Production Debugging

15 Chapter 15: Retrieval Tuning, Embeddings, Hybrid Search, and RAG Evaluation

15.1 Tune Similarity Thresholds, Chunk Sizes, and Retrieval Strategies 15.2 Select and Fine-Tune Embedding Models 15.3 Hybrid Search: Semantic and Keyword Retrieval 15.4 RAG Relevance Metrics and A/B Testing

16 Chapter 16: Fine-Tuning Methods, Synthetic Data, and Promotion to Production

16.1 Advanced Fine-Tuning Methods (LoRA, QLoRA, Full) 16.2 Create and Manage Synthetic Data for Fine-Tuning 16.3 Monitor, Optimize, and Promote Fine-Tuned Models

16.1 Advanced Fine-Tuning Methods (LoRA, QLoRA, Full)

Key Takeaways Explain This Topic AI

Microsoft Foundry (Azure OpenAI) fine-tuning uses low-rank adaptation (LoRA), a parameter-efficient update. Full-weight supervised fine-tuning is not the serverless OpenAI path.

QLoRA is not a documented Foundry OpenAI jobs-API option. Treat it as a possible recipe only on managed-compute custom stacks you control, after you verify the job config.

Fine-tune for stable style, JSON schema, instruction following, and tool calling. Use RAG for facts that change. Exhaust prompt engineering before you pay for a custom model.

Training files are JSONL in the Chat Completions messages schema, UTF-8 with a BOM, and under 512 MB. Jobs need at least 10 examples; Microsoft recommends starting with about 50 well-crafted rows and scaling to hundreds or thousands.

Standard training stays in the resource region with data-residency guarantees. Global copies data and weights to the training region for lower cost and faster queues. Developer training uses idle capacity with no residency or SLA.

Last updated: August 2026

Advanced Fine-Tuning Methods (LoRA, QLoRA, Full)

Quick Answer: Microsoft Foundry Azure OpenAI fine-tuning uses low-rank adaptation (LoRA) — a parameter-efficient update, not a full-weight rewrite. Use it for style, format, and tool calling , not to inject facts (that is retrieval-augmented generation (RAG) ). Training data is JSON Lines (JSONL) in Chat Completions format, UTF-8 with a byte-order mark (BOM), under 512 MB . Jobs need at least 10 examples; Microsoft recommends starting with 50 well-crafted ones and scaling to hundreds or thousands. Standard training stays in-region; Global copies data and weights for cheaper, faster queues.

Domain 5 of Exam AI-300 — Optimize generative AI systems and model performance (10–15%) — includes design and implement advanced fine-tuning methods . An MLOps engineer who treats fine-tuning as “upload a wiki and the model will memorize it” will fail both the exam and production. This section is about when to fine-tune, which method Foundry actually runs, and how to submit a job that will start.

Fine-tuning versus prompt engineering versus RAG

Fine-tuning is not the first customization lever. Microsoft’s fine-tuning considerations article ranks the cheaper controls first, and AI-300 expects you to do the same.

Prompt engineering (system message, few-shot examples, structured-output instructions) is cheapest. Start here. When the few-shot prompt grows so long that token cost and latency hurt, you have a candidate for fine-tuning: you embed those examples into weights so each request can be shorter.

RAG is how you supply knowledge that changes : policies, SKUs, ticket history, rate tables. Fine-tuning does not replace a search index. Facts baked into weights go stale, cannot cite sources, and are expensive to refresh.

Fine-tuning shines when the behavior is stable: brand tone, a rigid JSON schema, calling the right tools, classification labels, instruction following, or distillation (using a large model’s production traces to train a cheaper small model such as GPT-4.1-mini).

Microsoft is explicit about the good use cases: reducing prompt overhead, modifying style and tone, generating outputs in a specific schema, enhancing tool usage, helping a model use retrieved context, and optimizing smaller models. Combining fine-tuning with retrieval can teach the model to consume Azure AI Search hits (and ignore junk), but the documents themselves still live in the index. Fine-tuning is not a knowledge base.

Exam scenario: A claims copilot must emit a five-field JSON object and call get_claim_status with a claim ID. After two weeks of prompt engineering, the system message is 4,000 tokens of examples and still misses the schema about 15 percent of the time. Fine-tune gpt-4.1-mini with hundreds of JSONL conversations that demonstrate the schema and tool calls. Keep the policy handbook in RAG. Do not fine-tune on the handbook text hoping the model will “know” next quarter’s deductible.

Common trap: Fine-tuning to “teach the model our product catalog.” Catalogs change weekly. You will retrain constantly, still hallucinate SKUs, and skip groundedness. Use RAG (Chapter 15) for facts; fine-tune for the answer shape .

LoRA, QLoRA, and full-weight updates

Low-rank adaptation (LoRA) is the method Microsoft documents for Foundry OpenAI fine-tuning. Instead of updating every weight, LoRA injects small low-rank matrices that approximate the change to the original high-rank weight matrix. Only that smaller subset of parameters is trained during supervised adaptation. Training is faster and cheaper than rewriting the base model; the resulting custom model is still served as a Foundry deployment.

Quantized LoRA (QLoRA) further compresses the frozen base weights (commonly 4-bit) so adapters can train on smaller GPUs. QLoRA is a standard recipe in open-source parameter-efficient fine-tuning (PEFT) stacks. Microsoft does not document QLoRA as a Foundry OpenAI fine-tuning API option. There is no qlora=true hyperparameter on fine_tuning.jobs.create . If an exam item mentions QLoRA, map it to a managed-compute custom job whose framework you configured (Azure Machine Learning / Foundry managed compute, your VM quota, a PEFT trainer). Verify the job YAML or component before you claim QLoRA is in use. Do not tell a stakeholder that clicking Fine-tune on GPT-4.1 in the portal “is QLoRA.”

Full-weight supervised fine-tuning (SFT) updates most or all parameters. It needs large GPU quota, more data, and more risk of catastrophic forgetting. On Foundry, full-weight customization is not what the Azure OpenAI fine-tuning job does. Full updates appear only if you run a managed-compute custom training stack on models that allow it (open-source catalog models or your own weights).

  • Method What updates Foundry OpenAI serverless Managed-compute custom stack Typical use
  • Prompt / few-shot Nothing (weights frozen) Always available Not applicable First attempt
  • LoRA SFT Low-rank adapters Yes — documented path Yes, if the trainer uses PEFT/LoRA Style, format, tools

QLoRA LoRA on quantized frozen weights Not a documented API option Possible in custom PEFT jobs GPU-poor open-source fine-tunes

Full-weight SFT Most or all parameters No Possible when the framework and quota allow Deep rewrite of open-source weights

Direct preference optimization (DPO) Preferred vs rejected completions Yes, on listed models (GPT-4o, GPT-4.1 family) Depends on stack Tone or safety alignment after SFT

Reinforcement fine-tuning (RFT) Grader / reward signals Yes, on listed reasoning models (o4-mini; GPT-5 RFT is invitation-gated) Depends on stack Objective domains with a grader

You can stack techniques: run SFT first to teach the task, then DPO to prefer one style over another. During SFT, obsess over data quality. During DPO, supply clear chosen versus rejected pairs.

Serverless versus managed compute

Foundry offers two modalities for fine-tuning. Do not confuse them with inference deployment types.

Serverless uses Microsoft-managed capacity and consumption pricing. You do not need GPU quota. This is how you fine-tune Azure OpenAI / Foundry Models sold by Azure (GPT-4.1, GPT-4.1-mini, GPT-4.1-nano, GPT-4o, GPT-4o-mini, and listed open-source models on Global). Hyperparameter knobs are fewer than on a custom trainer. Microsoft’s classic overview cites consumption starting around $1.70 per million input tokens for serverless fine-tuning; always confirm the current Azure OpenAI pricing page before a cost review, because price lists move.

Managed compute runs training on your virtual machines (Azure Machine Learning compute or Foundry managed compute). You need VM quota. OpenAI models are not trained this way. You get more hyperparameters and can choose PEFT recipes (LoRA, and in custom containers possibly QLoRA or full updates).

For most AI-300 GenAIOps work, serverless LoRA SFT on GPT-4.1 or GPT-4.1-mini is the path. Use managed compute when the model card says the model is only fine-tunable on dedicated GPUs (for example certain Llama catalog models). Serverless is also the only place you get exclusive access to OpenAI base models.

Open-source models listed for Foundry SFT in public preview (Ministral-3B, Qwen-32B, Llama-3.3-70B-Instruct, gpt-oss-20b) are supported on Foundry resources and the new Foundry UI, typically on Global training, not on Standard regional training. Check the current model table before you design a landing zone.

Customization methods on Foundry OpenAI

The portal customization method depends on the selected base model:

Supervised fine-tuning (SFT): labeled input/output chat JSONL. Start here for almost every project.

Direct preference optimization (DPO): preferred versus non-preferred completions. Computationally lighter than classic reinforcement learning from human feedback (RLHF) because you do not fit a separate reward model. Available on GPT-4o and the GPT-4.1 family.

Reinforcement fine-tuning (RFT): grader or reward signals when there are many valid solution paths. o4-mini supports RFT. GPT-5 RFT is generally available but gated by invitation through your Microsoft account team. RFT is the wrong first choice for “make the JSON valid.”

You may fine-tune a previously fine-tuned OpenAI model ( base-model.ft-{jobid} ) on a new JSONL set — continuous fine-tuning , OpenAI models only. Use a suffix (up to 18 characters; Foundry names cannot contain dots) so iterations stay distinguishable.

Data format, size, and hyperparameters

Training and validation files must be JSONL in the Chat Completions messages schema, encoded UTF-8 with a BOM , each file under 512 MB . Roles are system , user , assistant , and optionally tool for tool-calling traces. Multi-turn conversations can sit on one JSONL line. Set "weight": 0 on assistant turns you do not want to train on, and "weight": 1 on the turns that matter (for example, skip a bland first reply and train only on the sarcastic rewrite).

Vision-capable bases (GPT-4o, GPT-4.1) accept image-plus-text user content in the same chat schema. Keep the same system message you will use in production in every training example. Microsoft warns that a different system message at inference is a documented reason a custom model “does not work.”

Jobs will not start with fewer than 10 training examples. Ten is not enough to change behavior. Microsoft’s guidance: start with 50 well-crafted examples , then grow to hundreds or thousands . Doubling a high-quality set can lift quality roughly linearly; a large dirty dump can make the model worse than the base. Seed quality beats volume.

Leave hyperparameters at portal defaults for the first job. n_epochs , batch_size , and learning_rate_multiplier can be left automatic ( -1 where the API allows). Documented knobs — do not invent unpublished universal numbers:

n_epochs : full passes over the training set. Auto is allowed. Too many epochs overfit (training loss falls while validation loss rises).

batch_size : examples per forward/backward pass. Larger batches often suit larger sets. When set to -1 , Microsoft calculates about 0.2 percent of the training set, capped at 256 . Defaults and maxima are base-model specific .

learning_rate_multiplier : scales the pre-training learning rate. If you leave auto, Microsoft suggests experimenting in 0.02 to 0.2 . Smaller values reduce overfitting.

Pass a seed so the same job parameters usually reproduce. Importing training files from Azure Blob Storage requires the storage account to allow public network access . If corporate policy forbids that, upload locally or use the SDK.

Training types: Standard, Global, Developer

These are training tiers. They are not the same as inference deployment types (section 16.3), though the names overlap. In the Python SDK you set them with extra_body ( trainingType : Standard , GlobalStandard , or developerTier ) on a current preview API version.

Training type Where it runs Data residency Cost / queue Use

Standard Current Foundry resource region In-region guarantee Higher; may queue Regulated data that must not leave the region

Global Capacity beyond the current region Data and weights are copied to the training region Cheaper, faster queue Default when residency is not required

Developer Idle capacity No residency guarantee Cheapest; jobs may be preempted and resumed; no latency or SLA Experiments only

If the exam asks which training type keeps training data in the resource region, the answer is Standard , not Global and not Developer. Microsoft recommends Global when residency is not a constraint because of cost and queue time.

Role-based access control (RBAC): Foundry Users may submit fine-tune jobs; Foundry Owner (or a custom role with Microsoft.CognitiveServices/accounts/deployments/write ) is required to deploy . Role display names were recently renamed from Azure AI User / Azure AI Owner; the role IDs did not change. Automatic deployment after a successful job is optional, OpenAI models only , and still needs Owner — it is not a production promotion gate (section 16.3).

Checklist before you click Submit:

You already failed to solve the task with a short prompt and with RAG for facts.

Base model and customization method (SFT / DPO / RFT) match the model card.

JSONL validates, has a BOM, and is under 512 MB.

Training and validation files are disjoint.

Training type matches residency policy.

Hyperparameters are defaults unless a prior job overfit or underfit.

Suffix and seed are set so you can tell v3 from v4.

Loading diagram... Choose prompt, RAG, LoRA SFT, or managed-compute custom training