10.3 End-to-End MLOps CI/CD & Automated Retraining

AWS Certified Machine Learning Engineer — Associate MLA-C01

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Introduction & Exam Strategy

1.1 AWS MLA-C01 Blueprint, Format & Strategy 1.2 The Machine Learning Engineering Lifecycle on AWS

2 Domain 1: Data Ingestion & Storage Architecture for ML

2.1 Ingesting Batch and Streaming Data for ML 2.2 High-Performance Storage for Training and Inference 2.3 Data Cataloging, Governance, and Querying

3 Domain 1: Feature Engineering & Data Transformation

3.1 Distributed Feature Engineering with AWS Glue & DataBrew 3.2 Interactive Data Preparation with SageMaker Data Wrangler 3.3 Feature Management with SageMaker Feature Store

4 Domain 1: Data Quality, Integrity & Ground Truth Labeling

4.1 Data Validation, Schema Drift, and Quality Rules 4.2 Handling Missing Values, Outliers & Class Imbalance 4.3 Data Labeling with SageMaker Ground Truth & Ground Truth Plus

5 Domain 2: Model Selection & Framework Architecture

5.1 SageMaker Built-in Algorithms & Selection Criteria 5.2 Custom Models with Script Mode and BYOC 5.3 Foundation Models, JumpStart, and Amazon Bedrock

6 Domain 2: SageMaker Model Training & Hyperparameter Optimization

6.1 SageMaker Training Jobs & Cost Optimization 6.2 Distributed Training Strategies & Architectures 6.3 Hyperparameter Optimization with Automatic Model Tuning (AMT)

7 Domain 2: Model Evaluation, Tracking & Model Registry

7.1 ML Evaluation Metrics & Validation Strategies 7.2 Experiment Tracking with SageMaker Experiments 7.3 Model Governance with SageMaker Model Registry

8 Domain 3: SageMaker Inference Options & Endpoint Architecture

8.1 Real-Time, Serverless, and Asynchronous Inference 8.2 High-Throughput Offline Inference with Batch Transform 8.3 Multi-Model Endpoints (MME) & Multi-Container Endpoints (MCE)

9 Domain 3: Deployment Strategies & Infrastructure Scaling

9.1 Production Deployment Patterns & Traffic Shifting 9.2 Endpoint Auto-Scaling & Hardware Acceleration

10 Domain 3: MLOps Pipelines & Workflow Orchestration

10.1 SageMaker Pipelines Core Architecture 10.2 Advanced Pipeline Patterns & Integration 10.3 End-to-End MLOps CI/CD & Automated Retraining

11 Domain 4: Production Model Monitoring with SageMaker Model Monitor

11.1 Data Quality & Model Quality Monitoring 11.2 Model Bias & Model Explainability Drift 11.3 Operational Observability with Amazon CloudWatch

12 Domain 4: Responsible AI, Bias Detection & Governance

12.1 Pre-Training and Post-Training Bias with SageMaker Clarify 12.2 Model Documentation & Auditing with Model Cards & Dashboard 12.3 Safety, Guardrails & Generative AI Governance

13 Domain 4: ML Security, Network Isolation & Cost Optimization

13.1 IAM Roles and Least Privilege for ML 13.2 Network Isolation, VPC Endpoints & Data Encryption 13.3 ML Cost Optimization & Resource Governance

10.3 End-to-End MLOps CI/CD & Automated Retraining

Key Takeaways Explain This Topic AI

Amazon EventBridge acts as the real-time event bus connecting MLOps components, triggering SageMaker Pipelines on S3 data drops, Model Registry status changes, or Model Monitor drift alarms.

SageMaker Pipelines is purpose-built for ML DAG orchestration with native Model Registry and Lineage tracking, whereas AWS Step Functions is designed for broad enterprise orchestration across disparate AWS services (Glue, Athena, EMR, Lambda).

AWS CodePipeline and AWS CodeBuild provide automated CI/CD pipelines that execute unit tests, data quality validations, pipeline DAG compilation, and infrastructure-as-code (CDK/CloudFormation) blue/green endpoint deployments.

Closed-loop automated retraining architectures continuously capture production inferences, detect drift via SageMaker Model Monitor, trigger CloudWatch alarms, and initiate automated retraining runs via EventBridge.

Model promotion follows a decoupled pattern: engineers or evaluators approve a model package in the Model Registry ( Approved ), which emits an EventBridge event that triggers the staging/production deployment pipeline.

Last updated: August 2026

End-to-End MLOps CI/CD & Automated Retraining

Moving machine learning from research prototypes to enterprise production requires robust Continuous Integration, Continuous Delivery, and Continuous Training (CI/CD/CT) pipelines. An automated MLOps architecture must react to real-world events—such as new training data arriving in Amazon S3, human approval in the Model Registry, or statistical data drift detected by SageMaker Model Monitor—and orchestrate multi-step deployment workflows safely.

For the AWS Certified Machine Learning Engineer — Associate (MLA-C01) exam, you must master the integration between Amazon EventBridge , SageMaker Pipelines , AWS Step Functions , and AWS CodePipeline / AWS CodeBuild , and understand how to architect closed-loop automated retraining systems .

1. Event-Driven MLOps Architectures with Amazon EventBridge

Amazon EventBridge serves as the central nervous system of AWS MLOps architectures. By monitoring state changes emitted by AWS services, EventBridge rules can trigger automated actions without requiring polling loops or manual intervention. +---------------------------------------------------------------------------------------------------+

| EVENT-DRIVEN MLOPS EVENTBRIDGE ARCHITECTURE |
| |
| EVENT PRODUCERS: |
| +--------------------------+ +---------------------------+ +------------------------------+ |
| | Amazon S3 (EventBridge) | | SageMaker Model Registry | | SageMaker Model Monitor | |
| | - New training data file | | - Package state changed | | - CloudWatch Drift Alarm | |
| | s3:ObjectCreated | | to "Approved" | | (Data / Concept Drift) | |
| +--------------------------+ +---------------------------+ +------------------------------+ |
| | | | |
| +-----------------------------+-------------------------------+ |
| | |
| v |
| [Amazon EventBridge Default Event Bus] |
| | |
| +-----------------------------+-------------------------------+ |
| | Event Pattern Filter: | Event Pattern Filter: | |
| | State == Approved | Source == aws.s3 | |
| v v v |
| TARGETS: +-------------------------+ +-------------------------+ +-------------------+ |
| | AWS CodePipeline | | SageMaker Pipeline | | Amazon SNS Alert | |
| | (Deploy Endpoint) | | (Trigger Retraining) | | (Notify On-Call) | |
| +-------------------------+ +-------------------------+ +-------------------+ |
+---------------------------------------------------------------------------------------------------+
Core EventBridge MLOps Event Patterns
Pattern 1: Triggering Automated Retraining on New S3 Data Arrival

When new batch data lands in an Amazon S3 bucket, S3 emits an Object Created event to EventBridge (requires S3 Event Notifications with EventBridge enabled): {

"source": ["aws.s3"],
"detail-type": ["Object Created"],
"detail": {
"bucket": {
"name": ["mlops-training-data-production"]
},
"object": {
"key": [{"prefix": "incoming/daily-batch/"}]
}
}
}

Target: Invokes sagemaker:StartPipelineExecution to launch the automated training pipeline DAG with the new S3 data URI passed as a parameter override.

Pattern 2: Triggering Deployment Pipeline on Model Approval

When an ML engineer or lead signs off on a model package in SageMaker Model Registry, changing its status from PendingManualApproval to Approved : {

"source": ["aws.sagemaker"],
"detail-type": ["SageMaker Model Package State Change"],
"detail": {
"ModelPackageGroupName": ["CustomerChurnModels"],
"ModelApprovalStatus": ["Approved"]
}
}

Target: Triggers an AWS CodePipeline deployment stage that compiles AWS CloudFormation or AWS CDK templates to update the production real-time endpoint using Blue/Green canary guardrails.

Pattern 3: Triggering Retraining on Model Monitor Drift Alarms

When SageMaker Model Monitor detects statistical feature drift (e.g., Kolmogorov-Smirnov test breach) or concept drift (accuracy drop), it sends metrics to Amazon CloudWatch. A CloudWatch Alarm enters the ALARM state, emitting an EventBridge event that kicks off automated model retraining.

2. Architectural Comparison: SageMaker Pipelines vs. AWS Step Functions

A critical decision on the MLA-C01 exam is selecting between SageMaker Pipelines and AWS Step Functions for workflow orchestration. +---------------------------------------------------------------------------------------------------+

| SAGEMAKER PIPELINES VS. AWS STEP FUNCTIONS |
| |
| +---------------------------------------+ +-----------------------------------------------+ |
| | SAGEMAKER PIPELINES | | AWS STEP FUNCTIONS | |
| +---------------------------------------+ +-----------------------------------------------+ |
| | - Purpose-built for ML workflows | | - General-purpose enterprise orchestrator | |
| | - Native Model Registry integration | | - Connects 200+ AWS services (Glue, Athena) | |
| | - Native Lineage & Experiment tracking| | - Complex business processes & IT automation | |
| | - Python SDK-native (Data Science UX) | | - JSON/YAML Amazon States Language (ASL) | |
| | - Serverless control plane ($0) | | - Pay per state transition | |
| | - ML-specific step types & caching | | - Long-running workflows (up to 1 year) | |
| +---------------------------------------+ +-----------------------------------------------+ |
+---------------------------------------------------------------------------------------------------+
Detailed Architectural Decision Matrix

Criteria SageMaker Pipelines AWS Step Functions

Primary Purpose End-to-end machine learning lifecycle (Data prep, train, tune, eval, register). Broad enterprise application workflows spanning multi-service architectures.

Developer Persona Data Scientists and ML Engineers using the SageMaker Python SDK. DevOps, Cloud Architects, and Backend Engineers using CDK, Terraform, or ASL.

ML Governance Native: Automatic Lineage Tracking, Model Registry, Experiments, and Drift baselines. Manual: Requires explicit API calls to SageMaker APIs in state task definitions.

Service Breadth SageMaker-centric (Processing, Training, Tuning, Transform, Model Registry, Lambda). 200+ AWS Services (AWS Glue, Amazon EMR, Amazon Athena, DynamoDB, Lambda, ECS).

Step Caching Native: Built-in CacheConfig skipping redundant compute based on input ETags. None: Must be custom-built using DynamoDB state lookups.

Cost Model Free orchestration control plane; pay only for underlying SageMaker compute instances. Pay-per-state transition (Standard Workflows) or duration (Express Workflows).

Where do Apache Airflow and AWS CodeDeploy fit? Amazon Managed Workflows for Apache Airflow (Amazon MWAA) is the managed option when an organization has standardized on open-source Apache Airflow DAGs to orchestrate data and ML jobs across AWS and third-party systems; choose MWAA when existing Airflow DAG investments must be preserved, and choose SageMaker Pipelines for ML-native governance (Lineage, Experiments, Model Registry). Within the developer-tools family, the blueprint also expects familiarity with AWS CodeDeploy for compute-layer deployment mechanics (EC2/ECS/Lambda blue-green and canary shifts); for SageMaker endpoints specifically, native Deployment Guardrails (Chapter 9) provide the equivalent blue/green, canary, and linear traffic shifting without CodeDeploy.

Tip

Exam Selection Rule:

If the scenario involves ML-native training, model tuning, evaluation, Model Registry, and data science Python SDK authoring , choose SageMaker Pipelines .

If the scenario involves enterprise-wide cross-service data pipelines spanning AWS Glue ETL, Amazon EMR clusters, Amazon Athena queries, and third-party SaaS before invoking ML, choose AWS Step Functions (or have Step Functions trigger a SageMaker Pipeline).

3. CI/CD Automation with AWS CodePipeline & AWS CodeBuild

Production ML systems require rigorous CI/CD to prevent buggy model inference code or malformed preprocessing scripts from breaking production endpoints. +---------------------------------------------------------------------------------------------------+

| END-TO-END MLOPS CI/CD PIPELINE ARCHITECTURE |
| |
| [1. SOURCE STAGE] (AWS CodeCommit / GitHub) |
| - Triggers on git push to 'main' branch |
| - Contains: preprocessing.py, train.py, pipeline_dag.py, cdk_infra/ |
| | |
| v |
| [2. BUILD & TEST STAGE] (AWS CodeBuild) |
| - Runs unit tests (pytest) & linter (flake8) |
| - Validates data schemas & compiles pipeline DAG definition |
| - Calls `pipeline.upsert()` to update DAG in SageMaker Control Plane |
| | |
| v |
| [3. TRAINING & REGISTRATION STAGE] (SageMaker Pipelines) |
| - Executes automated training DAG |
| - Preprocesses data -> Trains model -> Evaluates AUC |
| - Registers new model package in SageMaker Model Registry (PendingManualApproval) |
| | |
| v |
| [4. STAGING DEPLOYMENT & INTEGRATION TEST] (AWS CodePipeline / CDK) |
| - Deploys model to staging endpoint |
| - Executes synthetic load tests & inference regression checks |
| | |
| v |
| [5. MANUAL / AUTOMATED APPROVAL GATE] (Model Registry State Change -> Approved) |
| | |
| v |
| [6. PRODUCTION DEPLOYMENT WITH GUARDRAILS] (AWS CloudFormation / CDK) |
| - Executes Blue/Green Canary update (10% traffic for 15-min baking period) |
| - Monitors CloudWatch Rollback Alarms (ModelLatency, 5XXErrors) |
| - Full promotion to 100% on success; Auto-rollback on alarm |
+---------------------------------------------------------------------------------------------------+

Key Components in the CI/CD Pipeline:

AWS CodeBuild buildspec.yml : Automates testing and DAG compilation: version: 0.2

phases:

install:

runtime-versions:

python: 3.10

commands:

- pip install -r requirements.txt

pre_build:

commands:

pytest tests/unit_tests/
flake8 src/

build:

commands:

python pipelines/customer_churn/pipeline.py --upsert
python pipelines/customer_churn/pipeline.py --start

SageMaker Model Registry Approval as the Decoupling Boundary: The CI pipeline updates the code and trains the model. The CD pipeline deploys the model. The two are cleanly decoupled by the Model Package Approval Status in the Model Registry.

4. Closed-Loop Continuous Retraining Architecture

In real-world applications, models suffer from concept drift (statistical relationships between features and targets change) and data drift (input feature distributions shift over time). A mature MLOps system implements a fully automated closed-loop continuous retraining architecture. +---------------------------------------------------------------------------------------------------+

| CLOSED-LOOP AUTOMATED RETRAINING ARCHITECTURE |
| |
| +-----------------------------------+ |
| | Client Inference Requests | |
| +-----------------------------------+ |
| | |
| v |
| +-----------------------------------+ |
| | SageMaker Real-Time Endpoint | |
| +-----------------------------------+ |
| | |
| (SageMaker Data Capture: Payload Logging) |
| v |
| +-----------------------------------+ |
| | Amazon S3 Capture Bucket | |
| +-----------------------------------+ |
| | |
| v |
| +-----------------------------------+ |
| | SageMaker Model Monitor Job | |
| | (Hourly Baseline Comparison) | |
| +-----------------------------------+ |
| | |
| (Drift Threshold Breached) |
| v |
| +-----------------------------------+ |
| | CloudWatch Metric & Alarm | |
| +-----------------------------------+ |
| | |
| v |
| +-----------------------------------+ |
| | Amazon EventBridge Rule | |
| +-----------------------------------+ |
| | |
| (StartPipelineExecution API) |
| v |
| +-----------------------------------+ |
| | SageMaker Training Pipeline | |
| +-----------------------------------+ |
| | |
| (Model Evaluated & Registered) |
| v |
| +-----------------------------------+ |
| | SageMaker Model Registry | |
| +-----------------------------------+ |
| | |
| (Approved -> Blue/Green Canary) |
| v |
| +-----------------------------------+ |
| | Updated Real-Time Endpoint Fleet | |
| +-----------------------------------+ |
+---------------------------------------------------------------------------------------------------+

Step-by-Step Retraining Loop Breakdown:

Data Capture: The SageMaker Endpoint is configured with DataCaptureConfig to asynchronously record a percentage (e.g., 100%) of live request and response payloads to Amazon S3.

Drift Monitoring: A scheduled SageMaker Model Monitor job executes periodically (e.g., hourly or daily), comparing captured inference data against the baseline constraints generated during training.

Alarm Generation: If the drift metric (e.g., data quality drift or prediction drift) violates the baseline threshold, Model Monitor publishes custom metrics to Amazon CloudWatch , placing an alarm in the ALARM state.

Event Routing: Amazon EventBridge catches the CloudWatch alarm state transition and triggers the target SageMaker Pipeline via the StartPipelineExecution API.

Retraining & Validation: The pipeline ingests the newly labeled dataset, retrains the model, evaluates performance metrics on held-out test data, and registers the new candidate model in the Model Registry .

Safe Automated Promotion: Upon approval, a second EventBridge rule triggers a Blue/Green canary deployment with automated CloudWatch rollback alarms, safely updating the production endpoint with zero customer downtime.

10.3 End-to-End MLOps CI/CD & Automated Retraining

AWS Certified Machine Learning Engineer — Associate MLA-C01

Track your progress as you read

1 Chapter 1: Introduction & Exam Strategy

1.1 AWS MLA-C01 Blueprint, Format & Strategy 1.2 The Machine Learning Engineering Lifecycle on AWS

2 Domain 1: Data Ingestion & Storage Architecture for ML

3 Domain 1: Feature Engineering & Data Transformation

4 Domain 1: Data Quality, Integrity & Ground Truth Labeling

5 Domain 2: Model Selection & Framework Architecture

6 Domain 2: SageMaker Model Training & Hyperparameter Optimization

7 Domain 2: Model Evaluation, Tracking & Model Registry

8 Domain 3: SageMaker Inference Options & Endpoint Architecture

9 Domain 3: Deployment Strategies & Infrastructure Scaling

9.1 Production Deployment Patterns & Traffic Shifting 9.2 Endpoint Auto-Scaling & Hardware Acceleration

10 Domain 3: MLOps Pipelines & Workflow Orchestration

11 Domain 4: Production Model Monitoring with SageMaker Model Monitor

12 Domain 4: Responsible AI, Bias Detection & Governance

13 Domain 4: ML Security, Network Isolation & Cost Optimization

10.3 End-to-End MLOps CI/CD & Automated Retraining

Key Takeaways Explain This Topic AI

Last updated: August 2026

End-to-End MLOps CI/CD & Automated Retraining

1. Event-Driven MLOps Architectures with Amazon EventBridge

Pattern 2: Triggering Deployment Pipeline on Model Approval

Pattern 3: Triggering Retraining on Model Monitor Drift Alarms

2. Architectural Comparison: SageMaker Pipelines vs. AWS Step Functions

Criteria SageMaker Pipelines AWS Step Functions

Tip

3. CI/CD Automation with AWS CodePipeline & AWS CodeBuild

AWS CodeBuild buildspec.yml : Automates testing and DAG compilation: version: 0.2

python: 3.10

- pip install -r requirements.txt

4. Closed-Loop Continuous Retraining Architecture

Loading diagram... Closed-Loop Automated Retraining & CI/CD Deployment Flow