Study guide
3.5 Practitioner MLOps: Repeatability, Monitoring, and Retraining
AWS AI Practitioner Study Guide Exam
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: AIF-C01 Orientation and Official Source Control
1.1 Credential Scope, Code, and Search Language 1.2 Exam Format, Cost, Score, and Delivery 1.3 Official Exam Guide, Skill Builder, and Practice Workflow 1.4 Target Candidate Boundaries and Out-of-Scope Tasks 1.5 Domain Weights and Study Prioritization 1.6 Exam Policy, Retake, Results, and Recertification
2 Chapter 2: AI/ML Foundations and Use-Case Fit
2.1 AI, ML, Deep Learning, and Core Terminology 2.2 Supervised, Unsupervised, and Reinforcement Learning 2.3 Data Types, Labels, Structure, and Quality 2.4 Inference Patterns: Batch, Real-Time, and Embedded AI 2.5 AI Use-Case Fit and No-AI Decisions 2.6 Classification, Regression, Clustering, Forecasting, and Recommendation 2.7 AI/ML Foundations Case Lab
3 Chapter 3: ML Lifecycle, Metrics, and Practitioner MLOps
3.1 Data Collection, EDA, Preprocessing, and Feature Concepts 3.2 Training, Evaluation, Deployment, and Monitoring Lifecycle 3.3 Model Sources: Managed APIs, Open Source, and Custom Models 3.4 Evaluation Metrics and Business Metrics 3.5 Practitioner MLOps: Repeatability, Monitoring, and Retraining 3.6 SageMaker Lifecycle Service Map 3.7 ML Lifecycle Case Lab
4 Chapter 4: Generative AI Foundations and Inference Concepts
4.1 Foundation Models, LLMs, Transformers, and Modalities 4.2 Tokens, Context Windows, Embeddings, and Vector Search 4.3 Inference Parameters: Temperature, Top-p, and Output Controls 4.4 Hallucination, Grounding, and Context Quality 4.5 GenAI Use-Case Fit and Risk Triage 4.6 Prompt Injection, Data Leakage, and User Input Risk 4.7 Generative AI Foundations Case Lab
5 Chapter 5: Prompting, Model Selection, Customization, and Evaluation
5.1 Prompt Engineering Patterns and Business Quality 5.2 Zero-Shot, Few-Shot, Templates, and Instruction Design 5.3 Model Selection: Capability, Latency, Cost, and Risk 5.4 RAG vs Fine-Tuning vs Prompting vs Custom Models 5.5 Model Evaluation, Human Review, and Red-Team Feedback 5.6 Cost, Performance, and Throughput Decision-Making 5.7 Prompting and Model Selection Case Lab
6 Chapter 6: Amazon Bedrock, RAG, Agents, and Guardrails
6.1 Bedrock Core Concepts, Model Access, and Managed FM Choice 6.2 Knowledge Bases, RAG, Data Sources, and Grounding 6.3 AI Agents, Tools, Orchestration, and Business Workflows 6.4 Guardrails, Content Filters, Denied Topics, and Sensitive Data 6.5 Bedrock Model Evaluation, Monitoring, and Human Feedback 6.6 Bedrock Cost, Latency, Throughput, and Operational Fit 6.7 Bedrock, RAG, and Agents Case Lab
7 Chapter 7: AWS Managed AI/ML Services and SageMaker Map
7.1 Managed AI Services vs Foundation Model Apps vs Custom ML 7.2 Text, Language, Search, and Document AI Services 7.3 Vision, Speech, Contact Center, and Personalization Services 7.4 Amazon Quick, Amazon Q Developer, Kiro, and Practitioner Fit 7.5 SageMaker Canvas, Studio, Clarify, Autopilot, and Data Wrangler 7.6 Data Foundation Services: S3, Glue, OpenSearch, and Quick Sight 7.7 AWS AI Service Selection Case Lab
8 Chapter 8: Responsible AI, Human Review, and Safety
8.1 Fairness, Bias, Transparency, and Explainability 8.2 Privacy, Safety, Human Review, and Accountability 8.3 Guardrails, Clarify, Human Review, and Content Safety Controls 8.4 Responsible AI Risk Registers and Governance Workflows 8.5 Monitoring, Feedback, Drift, and Incident Response 8.6 Responsible AI Case Lab
9 Chapter 9: Security, Compliance, Governance, and Cost Controls
9.1 Shared Responsibility, IAM, and Least Privilege for AI 9.2 Encryption, Secrets, Networking, and Data Privacy 9.3 Prompt Injection, Data Exfiltration, and GenAI Threat Modeling 9.4 Logging, Monitoring, CloudTrail, CloudWatch, and Config 9.5 Compliance: Artifact, Config, Macie, and Policy Evidence 9.6 AI Cost Controls, Pricing, Throughput, and Budget Governance 9.7 Security and Governance Case Lab
10 Chapter 10: Integrated AWS AI Business Scenario Labs
10.1 Customer Support GenAI Assistant Lab 10.2 Document Intelligence and Compliance Review Lab 10.3 Personalization, Forecasting, and Fraud Detection Lab 10.4 Enterprise Search, RAG, and Knowledge Management Lab 10.5 Responsible AI and Security Review Board Lab 10.6 Cost, Performance, and Operations Review Lab 10.7 Full AIF-C01 Business Simulation
11 Chapter 11: Final Review, Exam Readiness, and Recertification
11.1 Final 30-Day AIF-C01 Study Plan 11.2 Official Practice Resources and Weak-Domain Remediation 11.3 90-Minute Exam Timing, Flagging, and Guessing Workflow 11.4 Test-Day Checklist: Online or Test Center 11.5 Post-Exam Results, Retake, and Recertification Plan 11.6 AWS AI Practitioner Final Mixed Review
3.5 Practitioner MLOps: Repeatability, Monitoring, and Retraining
Key Takeaways Explain This Topic AI
MLOps is the discipline of making ML work repeatable, governed, monitored, and improvable after launch.
Practitioners should understand artifacts, versions, approvals, monitoring, rollback, and retraining triggers without needing to build pipelines.
Repeatability protects the business because teams must know which data, code, parameters, model version, and approval produced a production model.
Retraining should be triggered by evidence such as drift, new labels, changed business rules, incidents, or scheduled refresh needs.
Last updated: October 2026
MLOps without overbuilding
MLOps combines ML development practices with operations, governance, automation, monitoring, and continuous improvement. For the AWS Certified AI Practitioner, this is a recognition topic. You should know why MLOps matters and what good controls look like, but you are not expected to build SageMaker Pipelines, implement CI/CD infrastructure, or tune training jobs in detail.
A model is not just a file. A production ML system includes data sources, preprocessing steps, feature definitions, training code or service configuration, model artifacts, evaluation results, approvals, deployment configuration, monitoring rules, logs, user feedback, and retraining decisions. If the team cannot explain which version is live and why it was approved, the organization has an operational risk.
Repeatability means the team can trace how a model was created and reproduce the process when needed. This does not always mean every training run gives bit-for-bit identical results, because some algorithms and distributed systems have randomness. It means the organization tracks enough information to understand the lineage: dataset version, code version, configuration, parameters, training environment, evaluation report, approver, deployment time, and rollback target.
MLOps concern Practitioner question AWS context to recognize
Versioning Which model, data, and code version is live? SageMaker Model Registry, S3 versioning, source control
Approval Who accepted the model for production? Model registry approval states, change management
Deployment How is the model served or consumed? SageMaker endpoints, batch transform, managed AI API, application integration
Monitoring What alerts if behavior changes? CloudWatch, open-source SageMaker AI monitoring solutions, application logs
- Governance Is access least privilege and auditable? IAM, CloudTrail, KMS, tagging, data governance
- Retraining What evidence triggers refresh? Drift, new labels, scheduled review, incidents
- Rollback What happens if the model fails? Previous model version, rules fallback, manual process
- Monitoring as an operating agreement
Monitoring should be defined before launch, not after complaints arrive. Technical teams may monitor latency, error rates, endpoint CPU or GPU utilization, failed invocations, and request volume. Data teams may monitor missing values, schema changes, feature distributions, and prediction drift. Business teams may monitor conversion, fraud loss, review volume, customer satisfaction, or false-positive complaints.
The practitioner should insist on owners for each metric. An alarm that no one reviews is theater. A dashboard with no threshold is weak. A model that collects feedback but never uses it is not improving. The operating agreement should say what is measured, where it is measured, who responds, how quickly they respond, and what action they can take.
Monitoring also includes responsible AI concerns. If outputs affect people differently across segments, the team needs a way to measure and address that. If a generative AI application produces unsafe or unsupported responses, the team needs logging, user feedback, guardrails, and escalation. If sensitive data appears in prompts or logs, security and privacy teams need retention and access controls.
Retraining triggers
Retraining is the process of updating a model with new or improved data. It should not happen randomly or only because a calendar says so. A calendar may be useful for review, but retraining should be tied to evidence and business need. Common triggers include performance degradation, detected drift, new labeled data, changed product catalogs, changed fraud patterns, policy changes, new regions, new customer segments, or a major incident.
Retraining can also create risk. A new model can perform worse than the old one, break segment fairness, change user behavior, increase cost, or fail compliance review. The team should compare the candidate model with the current production model, review business metrics, approve the change, deploy in a controlled way, and keep a rollback option. For high-risk workflows, human review and phased rollout are often appropriate. Retraining decision workflow
1. Detect drift, performance change, new labels, or business rule change.
2. Confirm the issue is not a data pipeline or application bug.
3. Prepare an updated dataset under approved governance controls.
4. Train or update the candidate model.
5. Evaluate against the current production model and segment requirements.
6. Approve, deploy, monitor, and keep rollback available.
Governance and security in the MLOps loop
MLOps is not only a data science practice. It intersects with security, compliance, and cost management. IAM should restrict who can access training data, invoke endpoints, approve models, update pipelines, and read logs. AWS KMS can protect data and artifacts with encryption keys. CloudTrail can help audit API activity. CloudWatch can collect logs and metrics. Tags can support ownership and cost reporting. Data governance tools such as Lake Formation can help control access to shared datasets.
Cost matters because ML workloads can scale quickly. Training jobs can use expensive compute. Real-time endpoints can run continuously. Foundation model calls can grow with token volume. Batch jobs can process more records than expected. A practitioner should ask for budget alarms, usage forecasts, and right-sized deployment patterns.
What good practitioner MLOps looks like
A healthy MLOps plan is understandable to business and technical stakeholders. It names the production model version, the data lineage, the approval path, the monitoring metrics, the responsible owner, the incident response path, and the retraining criteria. It also explains when the model should not be used.
A weak plan says the model is done after launch. It has no baseline, no owner, no drift monitoring, no feedback loop, no rollback plan, and no cost controls. In that situation, the practitioner should not approve expansion. The right next step is to define operating controls, narrow the use case, or keep the model in pilot until the team can run it responsibly.