Study guide
12.2 Model Documentation & Auditing with Model Cards & Dashboard
AWS Certified Machine Learning Engineer — Associate MLA-C01
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Introduction & Exam Strategy
1.1 AWS MLA-C01 Blueprint, Format & Strategy 1.2 The Machine Learning Engineering Lifecycle on AWS
2 Domain 1: Data Ingestion & Storage Architecture for ML
2.1 Ingesting Batch and Streaming Data for ML 2.2 High-Performance Storage for Training and Inference 2.3 Data Cataloging, Governance, and Querying
3 Domain 1: Feature Engineering & Data Transformation
3.1 Distributed Feature Engineering with AWS Glue & DataBrew 3.2 Interactive Data Preparation with SageMaker Data Wrangler 3.3 Feature Management with SageMaker Feature Store
4 Domain 1: Data Quality, Integrity & Ground Truth Labeling
4.1 Data Validation, Schema Drift, and Quality Rules 4.2 Handling Missing Values, Outliers & Class Imbalance 4.3 Data Labeling with SageMaker Ground Truth & Ground Truth Plus
5 Domain 2: Model Selection & Framework Architecture
5.1 SageMaker Built-in Algorithms & Selection Criteria 5.2 Custom Models with Script Mode and BYOC 5.3 Foundation Models, JumpStart, and Amazon Bedrock
6 Domain 2: SageMaker Model Training & Hyperparameter Optimization
6.1 SageMaker Training Jobs & Cost Optimization 6.2 Distributed Training Strategies & Architectures 6.3 Hyperparameter Optimization with Automatic Model Tuning (AMT)
7 Domain 2: Model Evaluation, Tracking & Model Registry
7.1 ML Evaluation Metrics & Validation Strategies 7.2 Experiment Tracking with SageMaker Experiments 7.3 Model Governance with SageMaker Model Registry
8 Domain 3: SageMaker Inference Options & Endpoint Architecture
8.1 Real-Time, Serverless, and Asynchronous Inference 8.2 High-Throughput Offline Inference with Batch Transform 8.3 Multi-Model Endpoints (MME) & Multi-Container Endpoints (MCE)
9 Domain 3: Deployment Strategies & Infrastructure Scaling
9.1 Production Deployment Patterns & Traffic Shifting 9.2 Endpoint Auto-Scaling & Hardware Acceleration
10 Domain 3: MLOps Pipelines & Workflow Orchestration
10.1 SageMaker Pipelines Core Architecture 10.2 Advanced Pipeline Patterns & Integration 10.3 End-to-End MLOps CI/CD & Automated Retraining
11 Domain 4: Production Model Monitoring with SageMaker Model Monitor
11.1 Data Quality & Model Quality Monitoring 11.2 Model Bias & Model Explainability Drift 11.3 Operational Observability with Amazon CloudWatch
12 Domain 4: Responsible AI, Bias Detection & Governance
12.1 Pre-Training and Post-Training Bias with SageMaker Clarify 12.2 Model Documentation & Auditing with Model Cards & Dashboard 12.3 Safety, Guardrails & Generative AI Governance
13 Domain 4: ML Security, Network Isolation & Cost Optimization
13.1 IAM Roles and Least Privilege for ML 13.2 Network Isolation, VPC Endpoints & Data Encryption 13.3 ML Cost Optimization & Resource Governance
12.2 Model Documentation & Auditing with Model Cards & Dashboard
Key Takeaways Explain This Topic AI
SageMaker Model Cards provide standardized, centralized, immutable documentation of model lineage, intended business uses, risk ratings, training hyperparameters, evaluation metrics, and ethical considerations.
Model Cards automatically ingest metadata from SageMaker Training Jobs, Model Registry Model Packages, and Clarify Processing Jobs, and can be enriched manually via Studio or programmatically via the Python SDK and boto3.
Model Cards support cryptographic PDF exports stored in Amazon S3 for regulatory compliance audits across frameworks such as EU AI Act, NIST AI RMF, SR 11-7, and FDA requirements.
SageMaker Model Dashboard serves as a centralized single-pane-of-glass governance portal tracking all deployed models, Model Registry packages, associated Model Cards, active Model Monitor schedules, and CloudWatch alarms across AWS accounts.
Model Dashboard automatically detects governance gaps, highlighting unmonitored endpoints lacking Data Quality, Model Quality, Bias, or Explainability monitors.
Last updated: August 2026
Model Documentation & Auditing with Model Cards & Dashboard
As enterprise machine learning systems scale into production across multiple business units, organizations face significant operational and regulatory risks. Models deployed without centralized documentation, clear lineage tracking, defined operating boundaries, and active production monitoring create severe compliance liabilities. Global regulatory frameworks—such as the EU AI Act , NIST Artificial Intelligence Risk Management Framework (NIST AI RMF) , Federal Reserve SR 11-7 Supervisory Guidance on Model Risk Management , and HIPAA/FDA clinical guidelines —mandate comprehensive, auditable documentation across the entire model lifecycle.
For the AWS Certified Machine Learning Engineer — Associate (MLA-C01) exam, you must master two key governance services in Amazon SageMaker: SageMaker Model Cards (standardized, auditable model documentation) and SageMaker Model Dashboard (centralized observability and governance monitoring across all account models).
1. Amazon SageMaker Model Cards Architecture & Schema
Amazon SageMaker Model Cards provide a standardized, immutable, and factory-integrated mechanism to record and share critical information about machine learning models throughout their development and production lifecycles. +--------------------------------------------------------------------------------------------------+
- | SAGEMAKER MODEL CARD FACTORY ARCHITECTURE |
- | |
- | [Automatic Metadata Ingestion] [Manual & Programmatic Enrichment] |
- | - SageMaker Training Job Metadata - Intended Uses & Business Problem |
- | (Algorithm, Instances, Hyperparameters) - Risk Rating (Low, Medium, High) |
- | - Model Registry Package Details - Target User Demographics & Domain Constraints |
- | (Model ARN, Framework Version, Approvals) - Ethical Considerations & Caveats |
- | - SageMaker Clarify Processing Jobs - Human-in-the-Loop Oversight Guidelines |
- | (Pre/Post-Training Bias, SHAP Metrics) |
- | | | |
- | v v |
- | +------------------------------------------------------------------------------------------+ |
- | | SAGEMAKER MODEL CARD | |
- | | Status: Draft | PendingReview | Approved | Archived | |
- | | Version: 1.0 (Immutable versioned audit trail in AWS Control Plane) | |
- | +------------------------------------------------------------------------------------------+ |
- | | | |
- | v v |
- | [SageMaker Model Dashboard] [Signed PDF Export in S3] |
- | (Single-pane-of-glass governance) (Tamper-evident audit document) |
- +--------------------------------------------------------------------------------------------------+
- Model Card Information Architecture
A complete SageMaker Model Card is structured into six core standardized sections: +--------------------------------------------------------------------------------------------------+
- | SAGEMAKER MODEL CARD CORE SECTIONS |
- | |
- | 1. MODEL OVERVIEW & INTENDED USES |
- | - Model Name, Version, Description, Owner, Creation Date |
- | - Intended Business Use Case & Domain Scope |
- | - Out-of-Scope Applications & Prohibited Use Cases |
- | |
- | 2. BUSINESS DETAILS & RISK RATING |
- | - Business Problem Formulation & Financial/Operational Value |
- | - Risk Rating: Low | Medium | High (Dictates approval gates & auditing frequency) |
- | |
- | 3. TRAINING DETAILS & LINEAGE |
- | - Ingestion Sources: S3 Dataset URIs, Schemas, Row Counts |
- | - Training Configuration: Algorithm (XGBoost/PyTorch), Instance Types, Hyperparameters |
- | - Training Metrics: Objective loss curves, Training accuracy, Epoch runtimes |
- | |
- | 4. EVALUATION & BENCHMARK RESULTS |
- | - Test Dataset Metadata & Validation Slicing |
- | - Performance Metrics: ROC-AUC, F1 Score, Precision, Recall, RMSE |
- | - Automated Clarify Ingestion: Bias Metrics (CI, DPL, DI) & Global SHAP Importance |
- | |
- | 5. ETHICAL CONSIDERATIONS & CAVEATS |
- | - Demographic Subgroup Performance Gaps & Mitigations |
- | - Known Edge-Case Failures & Performance Limitations |
- | |
- | 6. OPERATIONAL & HUMAN OVERSIGHT RECOMMENDATIONS |
- | - Human-in-the-Loop (HITL) Thresholds for Manual Review |
- | - Maintenance, Retraining Cadence, and Decommissioning Criteria |
- +--------------------------------------------------------------------------------------------------+
- Creation Workflows & Automation
Automated Population from SageMaker Resources:
When linked to an existing SageMaker Training Job , the model card automatically populates the algorithm container image, input dataset S3 locations, training instance types, hyperparameters, and environment variables.
When linked to a SageMaker Model Registry Package , the model card automatically imports artifact locations, approvals, and framework metadata.
When linked to a SageMaker Clarify Processing Job , the model card automatically imports pre-training bias metrics, post-training bias metrics, and SHAP feature attribution charts without manual data entry.
Manual Enrichment via SageMaker Studio: Data scientists and compliance officers can collaborate interactively in SageMaker Studio UI to author business descriptions, risk assessments, and intended use boundaries.
Programmatic Management via SDK / Boto3: Engineers can automate model card creation and updates within CI/CD pipelines using sagemaker.model_card or boto3 create_model_card .
Auditor PDF Export: The create_model_card_export_job API generates a cryptographically signed, professionally formatted PDF report stored in Amazon S3, enabling immediate compliance handoffs for internal risk committees or external regulatory inspectors.
2. Amazon SageMaker Model Dashboard
Amazon SageMaker Model Dashboard is a centralized governance portal embedded directly within SageMaker Studio that provides a single pane of glass to track, monitor, and govern all machine learning models across an AWS account and region. +--------------------------------------------------------------------------------------------------+
- | SAGEMAKER MODEL DASHBOARD INTERFACE |
- | |
- | Total Models: 42 | Endpoints Active: 18 | Governance Violations: 4 | Active Alerts: 2 |
- | +------------------------------------------------------------------------------------------+ |
- | | Model Name | Model Card | Registry | Data Quality | Model Quality | Bias | CloudWatch| |
- | | --------------- | ---------- | -------- | ------------ | ------------- | ---- | ----------| |
- | | fraud-xgb-prod | Approved | Approved | Healthy | Healthy | Healthy| 0 Alarms |
- | | churn-rf-v2 | Approved | Approved | Drift Alert! | Healthy | N/A | 1 ALARM |
- | | credit-risk-v1 | Missing! | Approved | Unmonitored! | Unmonitored! | N/A | 0 Alarms |
- | | recommender-v3 | Draft | Pending | Healthy | Drift Alert! | N/A | 1 ALARM |
- | +------------------------------------------------------------------------------------------+ |
- +--------------------------------------------------------------------------------------------------+
Core Governance Capabilities of Model Dashboard:
Comprehensive Inventory Tracking:
Automatically catalogs all SageMaker models, whether hosted on real-time endpoints, serverless endpoints, asynchronous endpoints, or registered in the Model Registry.
Maps deployed endpoints back to their underlying model packages, container images, and S3 artifact locations.
Governance Completeness Auditing:
Instantly highlights governance gaps , such as models operating in production without an associated Model Card or with a Model Card left in Draft state.
Identifies models with missing risk ratings or unapproved registry statuses.
Production Model Monitor Health Tracking:
Aggregates the scheduled execution status across all four SageMaker Model Monitor types:
Data Quality: Detects feature distribution drift against baseline constraints.
Model Quality: Detects accuracy, precision, and recall degradation against ground truth.
Bias Drift: Detects changes in demographic fairness metrics over live inference traffic.
Feature Attribution Drift: Detects shifts in SHAP feature importance relative to baseline.
Surfaces unmonitored models that are receiving live production traffic without active monitoring schedules.
CloudWatch Alarm Aggregation:
Directly consolidates CloudWatch alarms linked to endpoints (e.g., elevated ModelLatency , Invocation5XXErrors , or custom drift alarms), allowing ML engineers to triage operational incidents without navigating across multiple AWS consoles.
- 3. Python SDK Implementation: Programmatic Model Card Creation & Export import boto3
- import sagemaker
- from sagemaker.model_card import (
- ModelCard,
- ModelOverview,
- IntendedUses,
- BusinessDetails,
- TrainingDetails,
- EvaluationDetail,
- ModelCardStatusEnum,
- RiskRatingEnum,
- ObjectiveFunction
- )
- session = sagemaker.Session()
- role = sagemaker.get_execution_role()
- s3_bucket = session.default_bucket()
- # 1. Define Model Overview and Intended Use Boundaries
- overview = ModelOverview(
- model_name="credit-risk-assessment-v2",
- model_description="XGBoost binary classifier predicting loan default probability.",
- model_version=2,
- model_owner="mlops-governance-team@example.com",
- model_status=ModelCardStatusEnum.PENDING_REVIEW
- )
- intended_uses = IntendedUses(
- purpose_of_model="Automate credit risk scoring for retail personal loan applications under $50,000.",
- intended_uses="Underwrite primary consumer lending applications submitted through online web portal.",
- factors_affecting_model_efficiency="Economic recessions, rapid interest rate changes.",
- risk_rating=RiskRatingEnum.HIGH, # High risk due to financial/fair lending impact
- explanations_for_risk_rating="Model directly influences consumer credit approval decisions subject to ECOA."
- )
- # 2. Define Business Context
- business_details = BusinessDetails(
- business_problem="Reduce loan default rate by 15% while maintaining regulatory demographic parity.",
- business_stakeholders="Chief Risk Officer, Retail Lending VP, Fair Lending Compliance Officer"
- )
- # 3. Populate Training and Evaluation Details (Automatically links to SageMaker Training Job)
- training_details = TrainingDetails(
- objective_function=ObjectiveFunction(description="Minimize binary classification log-loss: binary:logistic"),
- training_job_details=None # Passing TrainingJobName auto-populates hyperparameters & S3 datasets
- )
- # 4. Instantiate and Populate the Model Card
- model_card = ModelCard(
- name="credit-risk-assessment-v2-card",
- sagemaker_session=session,
- model_overview=overview,
- intended_uses=intended_uses,
- business_details=business_details,
- training_details=training_details
- )
- # Automatically ingest metadata from completed Training and Clarify jobs
- model_card.load_training_job_details(training_job_name="credit-risk-xgb-training-2026-08")
- # 5. Persist the Model Card to SageMaker
- model_card.create()
- # 6. Generate an Audit-Ready PDF Export in S3 for Regulators
- export_job = model_card.export(
- s3_output_path=f"s3://{s3_bucket}/governance-audits/model-cards/",
- export_job_name="credit-risk-v2-compliance-export-2026"
- )
- print(f"Model card PDF export initiated: {export_job}")
- 4. Governance Comparison Table
Feature Dimension SageMaker Model Registry SageMaker Model Cards SageMaker Model Dashboard
Primary Purpose Versioning, artifact packaging, approval state transitions, and CI/CD deployment automation. Comprehensive, immutable, human/auditor-readable documentation of model lineage, risk, bias, and intended use. Centralized single-pane-of-glass operational and governance monitoring portal across all account models.
Primary Audience MLOps Engineers, DevOps, Automated Deployment Pipelines. Compliance Officers, Risk Committees, External Regulators, Auditors. ML Platform Engineers, Operations Teams, Governance Leads.
Key Artifacts ModelPackageGroup , ModelPackageArn , approval status ( Approved / Rejected ). ModelCard JSON object, cryptographically signed PDF reports in S3. Studio Governance Console, aggregated drift alarms, compliance gap views.
Automation Source CI/CD build scripts, SageMaker Pipelines RegisterModel step. Auto-populated from Training Jobs, Registry, and Clarify Processing Jobs. Continuous automated polling of account endpoints, monitors, and CloudWatch.