Study guide
2.2 High-Performance Storage for Training and Inference
AWS Certified Machine Learning Engineer — Associate MLA-C01
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Introduction & Exam Strategy
1.1 AWS MLA-C01 Blueprint, Format & Strategy 1.2 The Machine Learning Engineering Lifecycle on AWS
2 Domain 1: Data Ingestion & Storage Architecture for ML
2.1 Ingesting Batch and Streaming Data for ML 2.2 High-Performance Storage for Training and Inference 2.3 Data Cataloging, Governance, and Querying
3 Domain 1: Feature Engineering & Data Transformation
3.1 Distributed Feature Engineering with AWS Glue & DataBrew 3.2 Interactive Data Preparation with SageMaker Data Wrangler 3.3 Feature Management with SageMaker Feature Store
4 Domain 1: Data Quality, Integrity & Ground Truth Labeling
4.1 Data Validation, Schema Drift, and Quality Rules 4.2 Handling Missing Values, Outliers & Class Imbalance 4.3 Data Labeling with SageMaker Ground Truth & Ground Truth Plus
5 Domain 2: Model Selection & Framework Architecture
5.1 SageMaker Built-in Algorithms & Selection Criteria 5.2 Custom Models with Script Mode and BYOC 5.3 Foundation Models, JumpStart, and Amazon Bedrock
6 Domain 2: SageMaker Model Training & Hyperparameter Optimization
6.1 SageMaker Training Jobs & Cost Optimization 6.2 Distributed Training Strategies & Architectures 6.3 Hyperparameter Optimization with Automatic Model Tuning (AMT)
7 Domain 2: Model Evaluation, Tracking & Model Registry
7.1 ML Evaluation Metrics & Validation Strategies 7.2 Experiment Tracking with SageMaker Experiments 7.3 Model Governance with SageMaker Model Registry
8 Domain 3: SageMaker Inference Options & Endpoint Architecture
8.1 Real-Time, Serverless, and Asynchronous Inference 8.2 High-Throughput Offline Inference with Batch Transform 8.3 Multi-Model Endpoints (MME) & Multi-Container Endpoints (MCE)
9 Domain 3: Deployment Strategies & Infrastructure Scaling
9.1 Production Deployment Patterns & Traffic Shifting 9.2 Endpoint Auto-Scaling & Hardware Acceleration
10 Domain 3: MLOps Pipelines & Workflow Orchestration
10.1 SageMaker Pipelines Core Architecture 10.2 Advanced Pipeline Patterns & Integration 10.3 End-to-End MLOps CI/CD & Automated Retraining
11 Domain 4: Production Model Monitoring with SageMaker Model Monitor
11.1 Data Quality & Model Quality Monitoring 11.2 Model Bias & Model Explainability Drift 11.3 Operational Observability with Amazon CloudWatch
12 Domain 4: Responsible AI, Bias Detection & Governance
12.1 Pre-Training and Post-Training Bias with SageMaker Clarify 12.2 Model Documentation & Auditing with Model Cards & Dashboard 12.3 Safety, Guardrails & Generative AI Governance
13 Domain 4: ML Security, Network Isolation & Cost Optimization
13.1 IAM Roles and Least Privilege for ML 13.2 Network Isolation, VPC Endpoints & Data Encryption 13.3 ML Cost Optimization & Resource Governance
2.2 High-Performance Storage for Training and Inference
Key Takeaways Explain This Topic AI
SageMaker training input data modes offer distinct trade-offs: File Mode downloads the entire dataset to local EBS/NVMe storage before training begins; Fast File Mode mounts S3 as a virtual POSIX filesystem streaming chunks on demand with zero startup latency; Pipe Mode streams sequential records directly from S3 into memory using Linux FIFOs with zero disk consumption.
Amazon S3 Express One Zone is a purpose-built high-performance storage class located inside a single Availability Zone, delivering single-digit millisecond data access latency and up to 10x lower latency than S3 Standard for high-iteration model training.
Amazon FSx for Lustre delivers hundreds of gigabytes per second throughput and millions of IOPS with sub-millisecond latencies, synchronizing with Amazon S3 via Data Repository Associations with lazy loading (hsm_restore) on first file access.
Amazon EFS provides a serverless, shared POSIX-compliant network file system natively integrated with Amazon SageMaker Studio for multi-user workspace storage, shared custom Conda environments, and script collaboration.
Storage selection must balance cost, durability, and throughput: S3 Standard/Intelligent-Tiering for durable data lake storage; S3 Express One Zone or FSx for Lustre for high-throughput GPU training clusters; EFS for shared development environments; EBS gp3/io2 for temporary worker scratch disk.
Last updated: October 2026
High-Performance Storage for Training and Inference
Training deep neural networks and large-scale tabular models requires sustained high-throughput, low-latency access to training datasets. When GPU worker instances spend compute cycles waiting for data I/O from storage, training times increase and expensive accelerator hardware sits idle. Choosing the correct storage tier and data access mode in AWS directly impacts training cost, startup latency, and overall throughput.
1. Amazon SageMaker Training Input Data Modes
When launching a SageMaker training job via the Python SDK ( sagemaker.estimator.Estimator ), you configure how data is ingested from Amazon S3 into the training container via the input_mode parameter. +-----------------------------------------------------------------------------+
- | SAGEMAKER TRAINING INPUT DATA MODES |
- | |
- | 1. FILE MODE: |
- | [Amazon S3] ===(Download All Data Ahead of Time)===> [Local EBS Disk] |
- | | (Local Read) |
- | v |
- | [Training Script] |
- | |
- | 2. FAST FILE MODE: |
- | [Amazon S3] <---(On-Demand Chunk Streaming / POSIX Mount)--- [Training] |
- | |
- | 3. PIPE MODE: |
- | [Amazon S3] ===(Stream via Linux Named Pipe FIFO)===> [Memory / Stream] |
- +-----------------------------------------------------------------------------+
- Comprehensive Comparison of SageMaker Input Modes
Feature File Mode Fast File Mode (FFM) Pipe Mode
Mechanism Downloads entire S3 dataset to local EBS/NVMe disk before training starts Mounts S3 as read-only POSIX filesystem; streams file chunks on-demand Streams data directly from S3 into memory using Linux Named Pipes (FIFOs)
Startup Latency High (proportional to dataset size; e.g., 20+ mins for 500 GB) Near-zero (starts in seconds regardless of dataset size) Near-zero (starts streaming immediately)
Local Disk Requirement Must provision EBS volume equal to or greater than dataset size Minimal local disk required (uses local cache for accessed chunks) Zero local disk required for dataset storage
File System Access Standard local filesystem ( /opt/ml/input/data/<channel> ) Standard POSIX filesystem calls ( open() , os.listdir() , PIL.Image ) Reads sequentially from named pipe file stream ( open(fifo_path, 'rb') )
Random Seeking Fully supported (random access across all files) Supported (byte-range GETs fetched dynamically) Not Supported (sequential forward stream only)
Supported Formats Any file format Any file format RecordIO-protobuf, TFRecord, newline-delimited text/CSV
Code Modification None (standard file paths) None (standard file paths) Requires pipe reading logic or SageMaker built-in algorithm support
- Practical Code Example: Configuring Input Modes import sagemaker
- from sagemaker.inputs import TrainingInput
- from sagemaker.pytorch import PyTorch
- # Configure TrainingInput channel with Fast File Mode
- train_input = TrainingInput(
- s3_data="s3://ml-dataset-prod/imagenet-1k/",
- input_mode="FastFile", # Options: 'File', 'Pipe', 'FastFile'
- content_type="application/x-image"
- )
- # Define PyTorch Estimator
- estimator = PyTorch(
- entry_point="train.py",
- role=sagemaker.get_execution_role(),
- instance_count=4,
- instance_type="ml.p4d.24xlarge",
- framework_version="2.1.0",
- py_version="py310",
- # Notice: No need to allocate 2000 GB of expensive EBS storage
- volume_size=50
- )
estimator.fit({"train": train_input})
Note
Fast File Mode vs. Pipe Mode on the Exam: Fast File Mode is the modern recommendation for arbitrary file formats (images, audio, video, custom PyTorch/TensorFlow datasets) because it requires zero code changes while avoiding upfront download delays. Pipe Mode provides maximum streaming throughput for sequential record streams (RecordIO/CSV) and built-in SageMaker algorithms (e.g., BlazingText, Image Classification, XGBoost).
2. Amazon S3 Storage Classes for ML Workloads
Amazon S3 provides multiple storage tiers with distinct performance, access latency, and cost structures. +-----------------------------------------------------------------------------+
- | S3 STORAGE TIER SPECTRUM |
- | |
- | [LOWEST LATENCY / HIGH IOPS] [LOWEST COST ARCHIVE]|
- | |
- | S3 Express ---> S3 Standard ---> Intelligent- ---> Glacier |
- | One Zone (Multi-AZ) Tiering Deep Archive|
- | (Single-digit ms) (11 9s Durability) (Auto Cost Opt) (Long-term) |
- +-----------------------------------------------------------------------------+
- 1. Amazon S3 Express One Zone
Architecture: Purpose-built, high-performance storage class that colocates data within a single Availability Zone in dedicated directory buckets ( s3://bucket--az_id--x-s3/ ).
Performance: Single-digit millisecond data access latency; up to 10x lower latency and 50% lower request costs compared to S3 Standard.
ML Workload Fit:
High-frequency model checkpointing during distributed multi-node LLM training.
Storing hundreds of thousands of small image/audio files for immediate random access.
Eliminates S3 503 Slow Down throttling during intensive parallel training reads without provisioning external cache clusters.
2. Amazon S3 Standard & Intelligent-Tiering
S3 Standard: Default multi-AZ storage providing 99.999999999% (11 9s) durability. Ideal for active training data lakes, validated feature stores, and frequently queried datasets.
S3 Intelligent-Tiering: Automatically moves objects between three access tiers (Frequent, Infrequent [30 days], and Archive Instant Access [90 days]) based on changing access patterns with zero retrieval fees . Ideal for ML data lakes where research datasets have unpredictable access frequencies.
3. S3 Glacier Flexible & Deep Archive
S3 Glacier Flexible Archive (formerly Glacier): Retrieval times from minutes to hours; suitable for historical dataset snapshots and model audit logs.
S3 Glacier Deep Archive: Lowest-cost storage (standard retrievals within 12 hours, bulk within 48 hours); ideal for long-term regulatory compliance data retention (e.g., retaining clinical trial training inputs for 7–10 years).
3. Amazon FSx for Lustre for Distributed Deep Learning
When training massive deep learning models (computer vision, transformers, generative AI) across distributed GPU clusters (e.g., dozens of ml.p4d.24xlarge or ml.p5.48xlarge nodes), reading directly from S3 can encounter network bandwidth bottlenecks or API throttling. +-----------------------------------------------------------------------------+
- | AMAZON FSx FOR LUSTRE WITH S3 DATA REPOSITORY |
- | |
- | [Amazon S3 Dataset Bucket] (Petabytes of raw training images/audio) |
- | | |
- | | Data Repository Association (DRA) |
- | v |
- | [Amazon FSx for Lustre Cluster] (NVMe SSD / Sub-ms Latency) |
- | ├── Metadata Sync: All S3 files visible immediately |
- | ├── Lazy Loading: Byte-ranges loaded from S3 on first read (hsm_restore) |
- | └── Subsequent Epochs: Read 100% locally from Lustre NVMe cache (HPC Speed)|
- | | |
- | | POSIX Mount (`/opt/ml/input/data/training`) |
- | v |
- | [SageMaker Distributed GPU Training Cluster] (PyTorch FSDP / DeepSpeed) |
- +-----------------------------------------------------------------------------+
Architectural Features of FSx for Lustre:
Data Repository Association (DRA): Links an S3 bucket or prefix directly to the Lustre filesystem.
Lazy Loading (On-Demand Hydration): When FSx for Lustre is created, it instantly populates directory metadata (file names, sizes, permissions) without copying file contents. When a training worker opens a file, FSx streams the specific byte ranges from S3 on demand ( hsm_restore ).
Multi-Epoch Acceleration: In epoch 1, data streams from S3 through FSx. In epochs 2 through 100+, all reads are served entirely from the local FSx NVMe/SSD parallel storage tier at hundreds of GB/s throughput and sub-millisecond latency , maximizing GPU utilization.
Data Repository Tasks (Export Sync): Training checkpoints written to /fsx/checkpoints/ can be automatically or asynchronously synced back to Amazon S3 as durable objects.
Deployment Options:
Scratch File System: Designed for temporary storage and shorter-term training jobs. Data is not replicated across servers; provides the highest compute-to-cost ratio.
Persistent File System: Designed for longer-term workloads. Highly available with automatic replication within an Availability Zone. # Using FSx for Lustre with SageMaker Estimator via Python SDK
from sagemaker.inputs import FileSystemInput
- fsx_input = FileSystemInput(
- file_system_id="fs-0123456789abcdef0",
- file_system_type="FSxLustre",
- directory_path="/fsx/imagenet/",
- file_system_access_mode="ro" # Read-only or Read-write
- )
- estimator.fit({"training": fsx_input})
- 4. Amazon Elastic File System (EFS) for Shared ML Assets
Amazon EFS provides a serverless, fully elastic, multi-AZ POSIX-compliant Network File System (NFSv4). +-----------------------------------------------------------------------------+
- | AMAZON EFS IN SAGEMAKER STUDIO |
- | |
- | [SageMaker Studio Domain] (Multi-User Environment) |
- | ├── User A Studio Notebook \ |
- | ├── User B Studio Notebook ---> [Amazon EFS Volume (Multi-AZ)] |
- | └── User C Studio Notebook / ├── `/home/sagemaker-user/` |
- | ├── Custom Conda Virtualenvs |
- | └── Shared Git Repositories / Scripts |
- +-----------------------------------------------------------------------------+
Use Cases for Amazon EFS in ML:
SageMaker Studio User Workspaces: Serves as the persistent storage layer for SageMaker Studio domains. User home directories, custom Python packages, and Jupyter notebooks persist across kernel and instance lifecycle restarts.
Shared Project Assets: Enables multiple data scientists and distributed instances to share utility scripts, tokenizer vocabularies, and pre-computed embedding caches concurrently.
Throughput Modes: Configure Elastic Throughput for unpredictable notebook workloads, or Provisioned Throughput for dedicated batch preprocessing tasks.
5. Storage Architecture Comparison Matrix
Storage Service Protocol / Access Max Throughput / Latency Persistence / Durability Primary ML Role
Amazon S3 Standard REST API ( s3:// ) 100s of Gbps / 10–20 ms Multi-AZ / 11 9s Durability Scalable ML Data Lake, raw datasets, model artifacts
Amazon S3 Express One Zone REST API ( s3://...--az-id--x-s3 ) Single-digit ms latency / high IOPS Single-AZ / 11 9s Durability Ultra-fast training checkpoints, high-IOPS small file reads
Amazon FSx for Lustre POSIX ( /fsx/... ) 100s of GB/s / Sub-ms latency Scratch (temporary) or Persistent (single-AZ) Distributed deep learning on GPU clusters; S3 cache acceleration
Amazon EFS NFSv4 POSIX ( /mnt/efs/... ) Up to 10+ GB/s / Low ms latency Multi-AZ / 11 9s Durability SageMaker Studio user homes, shared notebooks, persistent virtualenvs
Amazon EBS (gp3/io2) Block storage Up to 2,000 MiB/s per gp3 volume (io2 Block Express goes higher) Single-AZ block volume Local scratch disk attached to individual training/inference instances
Important
Decision Framework for the MLA-C01 Exam:
If an exam question asks to train a model on TB-scale unstructured image data with minimal startup delay and no code modifications , select SageMaker Fast File Mode .
If the question involves multi-node GPU distributed deep learning across many epochs where S3 read latency is a bottleneck, select Amazon FSx for Lustre with S3 Data Repository Association .
If the question requires collaborative notebook storage shared across multiple SageMaker Studio users that persists across instance restarts, select Amazon EFS .