Study guide
3.5 Software Components Across the AI Development & Deployment Lifecycle
NVIDIA-Certified Associate: AI Infrastructure and Operations NCA-AIIO
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Exam Overview, AI Fundamentals & Adoption Drivers
1.1 Exam Structure, Logistics & Certification Roadmap 1.2 AI, Machine Learning & Deep Learning Workloads 1.3 Training vs. Inference Computational Profiles 1.4 Why AI Adoption Accelerated: Data, Compute & Algorithms 1.5 Key AI Use Cases Across Industries
2 Chapter 2: GPU Architecture & CUDA Fundamentals
2.1 GPU vs. CPU Architecture & Parallel Execution 2.2 CUDA Programming Model & Hierarchy 2.3 GPU Memory Hierarchy & Bandwidth
3 Chapter 3: NVIDIA AI Software Stack, Solution Portfolio & Lifecycle
3.1 CUDA-X Libraries & Accelerated Computing 3.2 NVIDIA AI Enterprise (NVAIE) & NeMo Framework 3.3 TensorRT, TensorRT-LLM & NIM Microservices 3.4 The NVIDIA Solution Portfolio: Domain Platforms & Blueprints 3.5 Software Components Across the AI Development & Deployment Lifecycle
4 Chapter 4: GPU Microarchitecture, Tensor Cores & Precision
4.1 Streaming Multiprocessors & Tensor Core Evolution 4.2 Numerical Precision Formats & Transformer Engine 4.3 High-Bandwidth Memory (HBM) & Memory Reliability
5 Chapter 5: Interconnects, NVLink & GPUDirect Technologies
5.1 PCIe, NVLink Generations & NVSwitch Architecture 5.2 GPUDirect Technologies: P2P, RDMA & Storage (GDS) 5.3 Distributed Training Interconnects & NCCL Collectives
6 Chapter 6: AI Systems, Networking, Facilities & Deployment Models
6.1 NVIDIA DGX Systems & HGX Architecture 6.2 AI Data Center Networking: InfiniBand & Spectrum-X 6.3 BlueField DPUs, Power, Cooling & SuperPOD Scale 6.4 On-Premises vs. Cloud AI Infrastructure & DGX Cloud 6.5 Data Center Facility Requirements & Site Readiness
7 Chapter 7: Workload Orchestration, Scheduling & Run:ai
7.1 AI Workload Scheduling: Slurm & Kubernetes 7.2 NVIDIA Run:ai Architecture & Dynamic Orchestration 7.3 NVIDIA Container Toolkit & Kubernetes GPU Operator
8 Chapter 8: GPU Virtualization, Partitioning & Multi-Instance GPU (MIG)
8.1 Multi-Instance GPU (MIG) Architecture & Profiles 8.2 NVIDIA vGPU Software & Virtualization Models 8.3 Multi-Tenant Workload Isolation & Resource Slicing
9 Chapter 9: AI Operations — Telemetry, DCGM, Triton Serving & Cluster Management
9.1 GPU Telemetry, nvidia-smi & DCGM Architecture 9.2 DCGM Health Checks, Diagnostics & Prometheus Monitoring 9.3 Triton Inference Server Architecture & Deployment 9.4 AI Data Center Management: Base Command Manager, BMC & Out-of-Band Networks
3.5 Software Components Across the AI Development & Deployment Lifecycle
Key Takeaways Explain This Topic AI
Blueprint objective 1.7 asks you to describe the software components tied to the life cycle of AI development and deployment, so the exam expects you to place a named tool in the correct lifecycle stage.
The canonical stages are data preparation, model development and training, experiment orchestration, optimisation, deployment and serving, and monitoring with feedback.
RAPIDS, DALI, and NeMo Curator accelerate the data stage; NGC framework containers, NeMo, and TAO Toolkit cover development and transfer learning.
TensorRT and TensorRT-LLM are optimisation-stage tools that compile a trained model into an inference engine — they never train, and confusing the two stages is a common exam trap.
Triton and NIM serve models, while DCGM plus Prometheus and Grafana close the loop with operational telemetry that feeds retraining decisions.
Last updated: August 2026
3.5 Software Components Across the AI Development & Deployment Lifecycle
Blueprint anchor: Domain 1 objective 1.7 — "Describe the software components related to the life cycle of AI development and deployment." The exam does not ask you to operate these tools; it asks you to place a named component in the right stage and to recognise which stage a described activity belongs to.
1. The Six-Stage Lifecycle
An enterprise AI system is never a single artefact. It is a loop that runs continuously, and each turn of the loop touches a different part of the software stack.
Stage What Happens NVIDIA / Ecosystem Components Artefact Produced
1. Data preparation Ingest, clean, deduplicate, label, transform, augment RAPIDS (cuDF, cuML), Spark RAPIDS accelerator, DALI (data loading/augmentation), NeMo Curator (LLM corpus curation) Curated, versioned dataset
2. Model development & training Architecture selection, pretraining, fine-tuning, alignment NGC framework containers (PyTorch, TensorFlow, JAX), NeMo , Megatron-Core , TAO Toolkit (transfer learning on pretrained vision/speech models) Trained checkpoint
3. Experiment orchestration Scheduling, quotas, distributed job launch, experiment tracking, checkpoint management Base Command Manager , Run:ai , Slurm, Kubernetes with the GPU Operator , MLflow/Weights & Biases Reproducible run history
4. Optimisation Quantise, prune, distil, fuse kernels, compile to an inference engine TensorRT , TensorRT-LLM , TensorRT Model Optimizer , ONNX export Optimised inference engine/plan
5. Deployment & serving Host the model, batch requests, scale, expose an API Triton Inference Server , NIM , NVIDIA AI Enterprise , Fleet Command (edge) Running inference endpoint
6. Monitoring & feedback Track hardware health, latency, throughput, drift; decide when to retrain DCGM + DCGM Exporter , Prometheus, Grafana, Triton metrics endpoint, NeMo Guardrails for output policy Telemetry, alerts, retraining trigger
The loop closes because stage 6 produces the evidence that sends you back to stage 1: drifting accuracy, new failure classes, or fresh production data worth labelling.
- 2. Stage Boundaries the Exam Actually Tests
- Training frameworks versus inference compilers
This is the single most commonly confused boundary. PyTorch, TensorFlow, and NeMo train. TensorRT and TensorRT-LLM do not train — they consume an already-trained model and compile it into a hardware-specific engine through layer fusion, kernel auto-tuning, precision calibration, and memory planning. If a stem describes "reducing latency of an existing model without changing its accuracy target," the answer lives in stage 4, not stage 2.
Optimisation versus serving
TensorRT produces an engine; Triton runs it. Triton is framework-agnostic — it has backends for TensorRT, ONNX Runtime, PyTorch, Python, and more — so "we need to serve models from three different frameworks behind one endpoint" is a Triton answer, while "we need this one model to run as fast as possible on H100" is a TensorRT answer.
Transfer learning versus pretraining
TAO Toolkit exists so that teams can adapt NVIDIA pretrained vision and speech models to their own classes with modest data and no deep-learning research effort. Pretraining a foundation model from scratch is a NeMo/Megatron-Core activity at cluster scale. A stem mentioning "a few thousand labelled images and no ML research team" is pointing at transfer learning.
Registry versus runtime
The NGC catalogue is where versioned containers and pretrained models come from ; it is not a runtime. Pinning an NGC container tag across a cluster is how you keep CUDA, driver, and framework versions consistent — a configuration-management answer, not a performance answer.
3. Why Containers Anchor the Whole Lifecycle
Every stage above ships as a container in practice, and that is deliberate:
Version coherence. A deep-learning stack couples the GPU driver, CUDA runtime, cuDNN, NCCL, the framework, and the model code. NGC images are built and tested as a matched set, so a job that ran in March runs identically in September.
Portability across stages and sites. The same image can run on a developer workstation, a Slurm batch node, a Kubernetes pod, and a cloud instance.
Reproducibility. An experiment is identified by image tag plus dataset version plus hyperparameters , which is what makes stage 3's run history meaningful.
Scheduling granularity. Because the unit of work is a container, orchestration (stage 3) and serving (stage 5) can use the same primitives — the NVIDIA Container Toolkit exposes GPUs to the container runtime, and the GPU Operator automates that plumbing across a Kubernetes cluster.
Exam cue: if a scenario mentions inconsistent results or dependency conflicts between nodes, the expected answer names standardised NGC containers, not a hardware change.