ExamFinch.com My Study

Study guide

5.3 Notebooks, Command Jobs, and Training Scripts

Practice this exam

Microsoft AI-300: Operationalizing Machine Learning and Generative AI Solutions Exam

Track your progress as you read

Finished sections get a check mark.

1 Introduction & Exam Overview

1.1 Current Exam Facts 1.2 Skills Measured, Format, and How This Guide Is Organized 1.3 Candidate Profile, Azure ML v2, Foundry, and DevOps Tooling

2 Chapter 2: Azure Machine Learning Workspaces, Datastores, Compute, and IAM

2.1 Create and Manage Azure Machine Learning Workspaces 2.2 Create and Manage Datastores 2.3 Create and Manage Compute Targets 2.4 Configure Identity and Access Management for Workspaces

3 Chapter 3: Azure Machine Learning Data Assets, Environments, Components, and Registries

3.1 Create and Manage Data Assets 3.2 Create and Manage Environments 3.3 Create and Manage Components 3.4 Share Assets Across Workspaces with Registries

4 Chapter 4: Infrastructure as Code, GitHub, and Network Isolation for Azure ML

4.1 GitHub Integration and Git Source Control for Machine Learning Projects 4.2 Deploy Workspaces and Resources with Bicep and Azure CLI 4.3 Automate Resource Provisioning with GitHub Actions 4.4 Restrict Network Access to Machine Learning Workspaces

5 Chapter 5: MLflow Tracking, AutoML, Training Jobs, and Hyperparameter Sweeps

5.1 Configure Experiment Tracking with MLflow 5.2 Use Automated Machine Learning to Explore Optimal Models 5.3 Notebooks, Command Jobs, and Training Scripts 5.4 Automate Hyperparameter Tuning

6 Chapter 6: Distributed Training, Training Pipelines, and Job Comparison

6.1 Manage Distributed Training for Large and Deep Learning Models 6.2 Implement Training Pipelines 6.3 Compare Model Performance Across Jobs

7 Chapter 7: Model Registration, Responsible AI, and Lifecycle Management

7.1 Register MLflow Models and Package Feature Retrieval Specs 7.2 Evaluate Models with Responsible AI Principles 7.3 Manage Model Lifecycle and Archiving

8 Chapter 8: Online Endpoints, Batch Endpoints, Rollout, and Rollback

8.1 Deploy Models to Managed Online Endpoints 8.2 Deploy Models to Batch Endpoints 8.3 Test, Troubleshoot, Progressive Rollout, and Safe Rollback

9 Chapter 9: Data Drift, Production Metrics, Retraining, and Alerts

9.1 Detect and Analyze Data Drift 9.2 Monitor Production Metrics and Trigger Retraining or Alerts

10 Chapter 10: Microsoft Foundry Resources, IAM, Networking, and IaC

10.1 Create and Configure Foundry Resources and Project Environments 10.2 Managed Identities and RBAC for Foundry 10.3 Foundry Network Security and Private Networking 10.4 Deploy Foundry Infrastructure with Bicep and Azure CLI

11 Chapter 11: Deploying and Managing Foundation Models

11.1 Serverless API Endpoints and Managed Compute for Foundation Models 11.2 Select Foundation Models for Specific Use Cases 11.3 Model Versioning and Production Deployment Strategies 11.4 Provisioned Throughput Units for High-Volume Workloads

12 Chapter 12: Prompt Design, Variants, and Git Version Control

12.1 Design and Develop Prompts 12.2 Create Prompt Variants and Compare Performance 12.3 Version Control for Prompts with Git

13 Chapter 13: Evaluation Datasets, Quality Metrics, Safety, and Automated Workflows

13.1 Test Datasets and Data Mapping for Evaluation 13.2 Groundedness, Relevance, Coherence, and Fluency 13.3 Risk and Safety Evaluations 13.4 Built-in and Custom Automated Evaluation Workflows

14 Chapter 14: Continuous Monitoring, Performance, Cost, Tracing, and Logging

14.1 Continuous Monitoring in Microsoft Foundry 14.2 Latency, Throughput, Token Use, and Cost Metrics 14.3 Logging, Distributed Tracing, and Production Debugging

15 Chapter 15: Retrieval Tuning, Embeddings, Hybrid Search, and RAG Evaluation

15.1 Tune Similarity Thresholds, Chunk Sizes, and Retrieval Strategies 15.2 Select and Fine-Tune Embedding Models 15.3 Hybrid Search: Semantic and Keyword Retrieval 15.4 RAG Relevance Metrics and A/B Testing

16 Chapter 16: Fine-Tuning Methods, Synthetic Data, and Promotion to Production

16.1 Advanced Fine-Tuning Methods (LoRA, QLoRA, Full) 16.2 Create and Manage Synthetic Data for Fine-Tuning 16.3 Monitor, Optimize, and Promote Fine-Tuned Models

5.3 Notebooks, Command Jobs, and Training Scripts

Key Takeaways Explain This Topic AI

Use compute-instance notebooks for exploration and debugging; production training is a command job (type: command) submitted from versioned files.

A command job YAML names code (the folder Azure Machine Learning snapshots), command, environment, optional compute, and inputs referenced as ${{inputs.<name>}}.

Omit the compute property to run on serverless compute; otherwise set compute: azureml:<cluster-or-instance>.

Submit with az ml job create -f job.yml (or SDK v2 command() + ml_client.jobs.create_or_update). Later edits to local files do not change a job that already snapshotted its code.

Untitled notebooks are not a production training surface: no code snapshot you can review, no repeatable environment pin, no clean experiment name for promotion.

Last updated: August 2026

Notebooks, Command Jobs, and Training Scripts

Quick Answer: Notebooks on a compute instance are for exploration. Production training is a command job ( type: command ) whose YAML sets code , command , environment , optional compute , and inputs . Submit with az ml job create -f job.yml . Azure Machine Learning snapshots the code folder at submit time. Do not train production models only in untitled notebooks.

Domain 2 clusters two official bullets: use notebooks for experimentation and exploration and run model training scripts . AI-300 expects you to know when each is legitimate — and why MLOps treats them as different surfaces.

Notebooks are the lab, not the factory

A compute instance is a managed development VM. You open Jupyter or VS Code against it, query data assets, plot distributions, test a 50-line scikit-learn fit, and call mlflow.start_run() as in section 5.1. That loop is how you discover whether a feature even belongs in the model. It is the wrong loop for a model you will register, deploy, and defend in an audit.

Problems with “we trained it in a notebook”:

The kernel state is not a code snapshot . A reviewer cannot re-run exactly those cells.

Untitled files have no Git history. Chapter 4 already required source control for Machine Learning projects.

Package versions drift with whatever the instance image happened to contain that morning unless you pin an environment .

Compute instances are one box. They do not autoscale like a cluster, and they keep billing while you leave them running.

Promotion scripts expect a job name whose outputs contain an MLflow model folder, not a casual model.pkl on the instance disk.

The healthy pattern is: explore in a notebook, then copy the working training path into src/main.py , parameterize it, and submit a command job. Keep the notebook as a report or as the place you debug a failing job by pulling the same inputs.

Anatomy of a command job

SDK/CLI v2 training is a job with type: command . A typical job.yml : $schema: https://azuremlschemas.azureedge.net/latest/commandJob.schema.json

  • code: src
  • command: >-
  • python main.py
  • -train-data ${{inputs.train_data}}
  • -learning-rate ${{inputs.learning_rate}}
  • environment: azureml:AzureML-sklearn-1.0-ubuntu20.04-py38-cpu@latest
  • compute: azureml:cpu-cluster

inputs:

train_data:

  • type: uri_folder
  • path: azureml:claims-features:2026-08-15
  • learning_rate: 0.05
  • experiment_name: claims-lgbm
  • display_name: claims-lgbm-baseline
  • Submit: az ml job create -f job.yml

SDK v2 is the same contract in Python: command(code="./src", command="python main.py --train-data ${{inputs.train_data}}", environment=..., inputs=..., compute="cpu-cluster") then ml_client.jobs.create_or_update(command_job) . To try serverless compute , omit compute (delete the YAML key or the compute= argument). Do not invent a compute named serverless .

Field Meaning Exam trap

code Local folder uploaded and snapshotted as a code asset Pointing at a single file and then importing a sibling that was not in the folder

command Exact process to run inside the environment Forgetting ${{inputs.*}} expressions and hard-coding /tmp paths

environment Curated azureml:<name>@latest or a custom conda-on-docker environment Assuming the compute instance’s interactive conda is what the job uses

compute Cluster or instance; omit for serverless Leaving a personal compute instance name in a production YAML

inputs Named values and data; files/folders use Input / YAML type + path Passing a datastore secret as a string input

experiment_name MLflow experiment grouping Relying on Default because the field was skipped

Expressions such as ${{inputs.train_data}} are SDK/CLI v2 substitutions resolved when the job starts. Your Python argparse then receives a local path (mount or download) or a URI, depending on the input mode you learned with data assets.

Snapshots, environments, and what the job actually runs

When you submit, Azure Machine Learning uploads the code directory and records that snapshot. The running job uses that copy. If you edit main.py on disk two minutes later, the in-flight job does not pick up the edit. That is a feature: reproducibility. It is also why “I fixed the bug but the job still failed” is usually a second submit, not a save in the editor.

The environment is the container plus Python packages — not “whatever was pip-installed on my compute instance last week.” Curated environments ( AzureML-lightgbm-* , AzureML-sklearn-* , and their successors) are the fast path. Custom environments are a base image plus a conda YAML, versioned as a workspace (or registry) asset. Jobs that point at an Azure Container Registry with a customized domain name label can fail on image pull; use the default *.azurecr.io login server.

Status walks Starting → Preparing → Running → Completed (or Failed/Canceled). Preparing includes image build and node allocation. Monitor with az ml job stream -n <name> , az ml job show --web , or ml_client.jobs.stream . Register an MLflow model from the job with a path such as azureml://jobs/{job_name}/outputs/artifacts/paths/model/ or az ml model create ... -p runs:/$run_id/model --type mlflow_model .

REST exists underneath both SDK and CLI ( jobType: Command ). AI-300 will not ask you to memorize Resource Manager JSON; it will ask you to know that CLI/SDK jobs are the supported authoring path and that a service principal token is required for unattended REST.

From notebook prototype to a training script

A production main.py should:

Parse hyperparameters and data paths from arguments (so a sweep in 5.4 can inject them).

Call mlflow.autolog() without start_run (the job already started a run).

Read inputs from the paths Azure Machine Learning mapped, not from a laptop ~/Downloads .

Log the primary metric under a stable name .

Write the MLflow model to a known artifact path.

Keep data preparation that is expensive out of every training restart when you can: a pipeline step (next chapter) or a registered data asset that already holds the features. Every command job rebuilds its environment and reloads data unless you designed otherwise.

GitHub Actions (Chapter 4) should call az ml job create -f job.yml on a branch or on main, not open a notebook. The YAML and src/ live in Git. The job name in studio is the audit trail.

Exam scenario

A data scientist has a working fraud classifier in Untitled12.ipynb on a compute instance, with pip install cells and a personal SAS token. The MLOps engineer copies the training cells into src/train.py , replaces the SAS path with azureml:claims-features:2026-08-15 , pins environment: azureml:<curated>@latest , writes job.yml with type implied by the command-job schema, and runs az ml job create -f job.yml against cpu-cluster (or omits compute for serverless). The notebook remains for exploratory plots. The registered model’s lineage points at the job , not at the notebook kernel.

Common trap

Do not train the production candidate only in an untitled notebook. Do not put compute: azureml:serverless — serverless is the absence of a compute property. Do not assume a job sees unsaved editor buffers; it sees the snapshot . Do not reuse the compute instance’s interactive conda as if it were the job environment. Do not hard-code /mnt/batch/tasks/... paths you copied from one log; use ${{inputs.*}} so the next cluster still works.