4.1 Data Validation, Schema Drift, and Quality Rules

AWS Certified Machine Learning Engineer — Associate MLA-C01

Track your progress as you read

Finished sections get a check mark.

1 Chapter 1: Introduction & Exam Strategy

1.1 AWS MLA-C01 Blueprint, Format & Strategy 1.2 The Machine Learning Engineering Lifecycle on AWS

2 Domain 1: Data Ingestion & Storage Architecture for ML

2.1 Ingesting Batch and Streaming Data for ML 2.2 High-Performance Storage for Training and Inference 2.3 Data Cataloging, Governance, and Querying

3 Domain 1: Feature Engineering & Data Transformation

3.1 Distributed Feature Engineering with AWS Glue & DataBrew 3.2 Interactive Data Preparation with SageMaker Data Wrangler 3.3 Feature Management with SageMaker Feature Store

4 Domain 1: Data Quality, Integrity & Ground Truth Labeling

4.1 Data Validation, Schema Drift, and Quality Rules 4.2 Handling Missing Values, Outliers & Class Imbalance 4.3 Data Labeling with SageMaker Ground Truth & Ground Truth Plus

5 Domain 2: Model Selection & Framework Architecture

5.1 SageMaker Built-in Algorithms & Selection Criteria 5.2 Custom Models with Script Mode and BYOC 5.3 Foundation Models, JumpStart, and Amazon Bedrock

6 Domain 2: SageMaker Model Training & Hyperparameter Optimization

6.1 SageMaker Training Jobs & Cost Optimization 6.2 Distributed Training Strategies & Architectures 6.3 Hyperparameter Optimization with Automatic Model Tuning (AMT)

7 Domain 2: Model Evaluation, Tracking & Model Registry

7.1 ML Evaluation Metrics & Validation Strategies 7.2 Experiment Tracking with SageMaker Experiments 7.3 Model Governance with SageMaker Model Registry

8 Domain 3: SageMaker Inference Options & Endpoint Architecture

8.1 Real-Time, Serverless, and Asynchronous Inference 8.2 High-Throughput Offline Inference with Batch Transform 8.3 Multi-Model Endpoints (MME) & Multi-Container Endpoints (MCE)

9 Domain 3: Deployment Strategies & Infrastructure Scaling

9.1 Production Deployment Patterns & Traffic Shifting 9.2 Endpoint Auto-Scaling & Hardware Acceleration

10 Domain 3: MLOps Pipelines & Workflow Orchestration

10.1 SageMaker Pipelines Core Architecture 10.2 Advanced Pipeline Patterns & Integration 10.3 End-to-End MLOps CI/CD & Automated Retraining

11 Domain 4: Production Model Monitoring with SageMaker Model Monitor

11.1 Data Quality & Model Quality Monitoring 11.2 Model Bias & Model Explainability Drift 11.3 Operational Observability with Amazon CloudWatch

12 Domain 4: Responsible AI, Bias Detection & Governance

12.1 Pre-Training and Post-Training Bias with SageMaker Clarify 12.2 Model Documentation & Auditing with Model Cards & Dashboard 12.3 Safety, Guardrails & Generative AI Governance

13 Domain 4: ML Security, Network Isolation & Cost Optimization

13.1 IAM Roles and Least Privilege for ML 13.2 Network Isolation, VPC Endpoints & Data Encryption 13.3 ML Cost Optimization & Resource Governance

4.1 Data Validation, Schema Drift, and Quality Rules

Key Takeaways Explain This Topic AI

AWS Glue Data Quality uses Data Quality Definition Language (DQDL) declarative rules to assess datasets against completeness, uniqueness, referential integrity, and custom SQL conditions.

PyDeequ is an open-source Python wrapper for AWS DeeQu built on Apache Spark, enabling programmatic data unit testing, constraint verification suites, constraint suggestion, and metrics repositories for ML pipelines.

AWS Glue Data Quality computes an automated quality score (0–100%) and publishes CloudWatch metrics ( glue.data.quality.rules.passed and glue.data.quality.rules.failed under the custom Glue Data Quality namespace) to monitor data pipelines over time.

In production MLOps pipelines, failing data quality rules can trigger conditional branching in AWS Glue or SageMaker Pipelines to halt model training and route defective records to an S3 quarantine prefix.

Schema drift detection in AWS Glue leverages catalog crawlers and Glue ETL schema evaluation to identify newly added, deleted, or type-altered columns before raw data reaches downstream feature stores or training jobs.

Last updated: August 2026

Data Validation, Schema Drift, and Quality Rules

In enterprise machine learning systems, data quality directly dictates model performance. While software engineering pipelines fail loudly when encountering syntax or network errors, machine learning systems often fail silently : corrupted, missing, or drifted data will still flow into training jobs and inference endpoints, producing degraded models and unreliable predictions without throwing runtime exceptions. This failure mode is captured by the foundational ML principle: Garbage In, Garbage Out .

For the AWS Certified Machine Learning Engineer — Associate (MLA-C01) examination, you must master automated data quality validation, declarative constraint authoring using AWS Glue Data Quality and Data Quality Definition Language (DQDL) , Spark-based data unit testing with PyDeequ , automated quarantine routing, and metric alerting via Amazon CloudWatch . +---------------------------------------------------------------------------------------------------+

• | AWS DATA QUALITY & VALIDATION ARCHITECTURE |
• | |
• | [Raw S3 Data] ---> [AWS Glue ETL / PySpark] |
• | | |
• | +---> [DQDL / PyDeequ Evaluation Engine] |
• | | |
• | +--------------------+--------------------+ |
• | | Score >= Threshold | Score < Threshold |
• | v v |
• | [Curated S3 Bucket] [Quarantine S3 Bucket] |
• | | | |
• | v v |
• | [SageMaker Training Job] [CloudWatch Metric & Alarm] |
• | | |
• | v |
• | [EventBridge Alert / Halt] |
• +---------------------------------------------------------------------------------------------------+
• 1. The Critical Role of Data Quality in Production ML

Data quality degradation manifests in multiple forms across the ML lifecycle:

Silent Data Degradation: Upstream source schema updates (e.g., changing a currency format or renaming a column) do not crash ingestion jobs if null values are injected by default, but cause models to make erratic inferences.

Schema Drift: Unannounced modifications to incoming data contracts, including newly introduced categorical values, altered column types, or dropped attributes.

Distribution Drift: Statistical properties of features change over time (e.g., mean income shifts due to inflation or geographic expansion), eroding the validity of previously trained parameters.

Target & Feature Leakage: Inadvertently including post-event indicators or corrupted ground truth labels in feature sets.

To prevent silent corruption, ML engineers implement automated quality gates that evaluate datasets against deterministic constraints prior to feature store ingestion or model training.

2. AWS Glue Data Quality & DQDL

AWS Glue Data Quality is a fully managed, serverless capability that measures and monitors data quality for data stored in Amazon S3, AWS Glue Data Catalog tables, and within active AWS Glue ETL PySpark pipelines.

Glue Data Quality uses Data Quality Definition Language (DQDL) , a declarative domain-specific language used to express data quality rules in a human-readable format. +---------------------------------------------------------------------------------------------------+

• | DQDL RULE TYPES & SYNTAX |
• | |
• | Completeness: IsComplete 'customer_id' |
• | Uniqueness: IsUnique 'transaction_uuid' |
• | Range / Bounds: ColumnValues 'customer_age' between 18 and 120 |
• | String Length: ColumnLength 'country_iso_code' == 2 |
• | Set Membership: ColumnValues 'subscription_tier' in ['FREE', 'BASIC', 'PREMIUM'] |
• | Cardinality: DistinctValuesCount 'device_type' <= 10 |
• | Data Type: ColumnDataType 'order_timestamp' = 'TIMESTAMP' |
• | Volume Gate: RowCount > 10000 |
• | Custom SQL: CustomSql 'SELECT count(*) FROM primary WHERE transaction_amt < 0' = 0 |
• +---------------------------------------------------------------------------------------------------+
• Core DQDL Rule Categories

• DQDL Rule Purpose Example DQDL Syntax
• IsComplete Validates that a column contains no null, empty, or whitespace-only values IsComplete 'customer_id'
• IsUnique Asserts that every value in a primary or foreign key column is distinct IsUnique 'order_id'

ColumnValues Enforces value ranges, comparison operators, or set membership ColumnValues 'age' between 18 and 120 ; ColumnValues 'state' in ['CA', 'NY', 'TX']

ColumnLength Restricts string length to fixed or bounded character counts ColumnLength 'zip_code' == 5

DistinctValuesCount Guards against unexpected cardinality explosion in categorical features DistinctValuesCount 'payment_method' <= 6

RowCount Validates that the ingested dataset meets minimum/maximum record thresholds RowCount between 10000 and 5000000

CustomSql Executes arbitrary SQL logic across the dynamic frame for complex cross-column validation CustomSql 'SELECT count(*) FROM primary WHERE discount_pct > 1.0' = 0

ReferentialIntegrity Verifies foreign key consistency between two distinct datasets ReferentialIntegrity 'user_id' 'reference_db.users.id' >= 0.99

Automatic Recommendation Engine

AWS Glue Data Quality includes a built-in recommendation engine. By analyzing historical dataset profiles, Glue automatically generates a starter DQDL ruleset based on observed distributions, null rates, and column types. ML engineers can review, customize, and version-control these generated DQDL rulesets.

• Evaluating DQDL within AWS Glue ETL Jobs
• In PySpark Glue jobs, DQDL rules are evaluated dynamically using the EvaluateDataQuality transform: import sys
• from awsglue.transforms import *
• from awsglue.utils import getResolvedOptions
• from pyspark.context import SparkContext
• from awsglue.context import GlueContext
• from awsglue.job import Job
• from awsglueml.transforms import EvaluateDataQuality

• sc = SparkContext()
• glueContext = GlueContext(sc)
• job = Job(glueContext)

• # 1. Load source dynamic frame
• source_dyf = glueContext.create_dynamic_frame.from_catalog(
• database='ml_lake',
• table_name='raw_sensor_telemetry'
• )

• # 2. Define DQDL ruleset
• dqdl_rules = '''
• Rules = [
• RowCount > 1000,
• IsComplete 'sensor_id',
• IsComplete 'timestamp',
• ColumnValues 'temperature_celsius' between -50.0 and 150.0,
• ColumnValues 'vibration_hz' >= 0.0,
• CustomSql 'SELECT count(*) FROM primary WHERE sensor_id IS NULL OR timestamp IS NULL' = 0
• ]
• '''

• # 3. Evaluate Data Quality with publishing enabled
• eval_results = EvaluateDataQuality.apply(
• frame=source_dyf,
• ruleset=dqdl_rules,
• publishing_options={
• 'dataQualityEvaluationContext': 'SensorETLJob',
• 'enableDataQualityCloudWatchMetrics': True,
• 'enableDataQualityResultsS3Publishing': True,
• 'dataQualityResultsS3Path': 's3://ml-lake-governance/data-quality-results/'
• }
• )

job.commit()

Note

Data Quality Score (0–100%): AWS Glue Data Quality aggregates rule evaluations into an overall quality score. If a ruleset contains 10 rules and 9 pass, the resulting Data Quality Score is 90% . You can enforce threshold gates (e.g., fail the job if Score < 95%).

3. PyDeequ & DeeQU on AWS

Deequ is an open-source library built on top of Apache Spark by AWS to define 'unit tests for data'. PyDeequ provides a native Python API for PySpark environments, making it a standard tool for ML engineers running data validation on Amazon EMR, AWS Glue, or Amazon SageMaker Processing jobs. +---------------------------------------------------------------------------------------------------+

• | PYDEEQU CORE MODULE ARCHITECTURE |
• | |
• | [PyDeequ Core Modules] |
• | 1. Suggestion: Suggests constraints automatically based on statistical profiling. |
• | 2. Verification: Runs test suites against Spark DataFrames; returns PASS/FAIL states. |
• | 3. Metrics Repo: Persists computed metrics over time in S3 / DynamoDB to detect drift. |
• | 4. Anomaly: Applies statistical models to metrics history to flag anomalous shifts. |
• +---------------------------------------------------------------------------------------------------+
• Core PyDeequ Capabilities
• 1. Verification Suite

The VerificationSuite executes declarative assertions against Spark DataFrames and returns a structured verification report: from pydeequ.checks import Check, CheckLevel, ConstrainableDataTypes

from pydeequ.verification import VerificationSuite, VerificationResult

• # Construct declarative verification check
• check = Check(spark, CheckLevel.Error, 'ML Training Dataset Verification')

• check_suite = (
• check.hasSize(lambda sz: sz >= 5000)
• .isComplete('customer_id')
• .isUnique('customer_id')
• .isContainedIn('subscription_status', ['active', 'churned', 'suspended'])
• .hasMin('credit_score', lambda val: val >= 300)
• .hasMax('credit_score', lambda val: val <= 850)
• .hasCompleteness('annual_income', lambda c: c >= 0.98) # Allow up to 2% missing
• .hasCorrelation('monthly_spend', 'credit_limit', lambda corr: corr > 0.3)
• )

• # Run suite
• verification_result = VerificationSuite(spark) \
• .onData(df) \
• .addCheck(check_suite) \
• .run()

• # Inspect verification results
• result_df = VerificationResult.checkResultsAsDataFrame(spark, verification_result)
• result_df.show(truncate=False)
• 2. Constraint Suggestion

When onboarding a new dataset without predefined constraints, PyDeequ analyzes the DataFrame and suggests suitable rules: from pydeequ.suggestions import ConstraintSuggestionRunner, DEFAULT

• suggestion_result = ConstraintSuggestionRunner(spark) \
• .onData(raw_spark_df) \
• .addConstraintRule(DEFAULT()) \
• .run()

for suggestion in suggestion_result['constraint_suggestions']:

• print(f"{suggestion['column_name']}: {suggestion['description']}")
• 3. Metrics Repository & Anomaly Detection

PyDeequ can store computed metrics in a Metrics Repository (persisted to an S3 bucket in JSON format). Successive pipeline runs compare current metrics against historical distributions to detect anomalies (e.g., sudden jumps in missing value proportions or feature variance changes): from pydeequ.repository import FileSystemMetricsRepository, ResultKey

• metrics_file = 's3://ml-lake-governance/deequ-metrics-repository.json'
• repository = FileSystemMetricsRepository(spark, metrics_file)
• result_key = ResultKey(spark, ResultKey.current_milli_time(), {'run_id': 'daily_etl_20260816'})

• VerificationSuite(spark) \
• .onData(df) \
• .addCheck(check_suite) \
• .useRepository(repository) \
• .saveOrAppendResult(result_key) \
• .run()
• 4. Automated Quality Gates, CI/CD & Quarantine Routing

Production ML architectures require automated mechanisms to handle records that violate quality constraints without crashing the entire business pipeline. +---------------------------------------------------------------------------------------------------+

• | DATA ROUTING & QUARANTINE PATTERN |
• | |
• | [Raw Input Batch] ---> [Data Quality Validation Rule Split] |
• | | |
• | +----------------+----------------+ |
• | | Passed Records | Failed Records |
• | v v |
• | [Curated Data (S3)] [Quarantine Buffer (S3)] |
• | | | |
• | v v |
• | [SageMaker Pipeline Step] [Glue Error Table & SNS Alert] |
• +---------------------------------------------------------------------------------------------------+
• The Quarantine Pattern

Record-Level Splitting: Incoming dynamic frames are split based on rule outcomes. Records passing all validation checks flow to the Curated S3 Prefix ( s3://lake/curated/year=YYYY/month=MM/ ).

Quarantine Storage: Records failing any check are routed to an Isolated Quarantine Prefix ( s3://lake/quarantine/year=YYYY/month=MM/ ) tagged with the failed rule identifier, timestamp, and source job ID.

Conditional Job Execution:

If the percentage of quarantined records exceeds an operational tolerance (e.g., > 3%), the ETL job raises an exception and triggers an alert.

Downstream SageMaker Pipelines utilize ConditionStep to verify the Data Quality status before launching model training. # PySpark dynamic routing pattern

• clean_df = df.filter('customer_id IS NOT NULL AND customer_age >= 18 AND customer_age <= 120')
• quarantine_df = df.filter('customer_id IS NULL OR customer_age < 18 OR customer_age > 120')

• # Write clean data for ML ingestion
• clean_df.write.mode('append').parquet('s3://ml-feature-store-lake/curated/customers/')

• # Route corrupted data for inspection
• quarantine_df.write.mode('append').json('s3://ml-feature-store-lake/quarantine/customers/')
• 5. CloudWatch Observability & Metric Alarms

When AWS Glue Data Quality executes with CloudWatch publishing enabled, it emits a pair of metrics under the custom CloudWatch namespace titled Glue Data Quality : +---------------------------------------------------------------------------------------------------+

• | CLOUDWATCH METRIC OBSERVABILITY MATRIX |
• | |
• | Metric Name Namespace Significance |
• | -------------------------------- ----------------- ------------------------------------- |
• | glue.data.quality.rules.passed Glue Data Quality Count of rules passed per evaluation |
• | glue.data.quality.rules.failed Glue Data Quality Count of rules failed per evaluation |
• +---------------------------------------------------------------------------------------------------+
• Incident Response Architecture

CloudWatch Alarm: Configured to trigger when glue.data.quality.rules.failed > 0 over a single evaluation period (any failed rule). The per-run Data Quality Score is available in the evaluation results emitted to S3/CloudWatch for threshold-based dashboards.

Amazon EventBridge: Captures the Glue Data Quality Rule Evaluation Failed state change event.

Remediation Actions: EventBridge invokes an AWS Lambda function that disables the downstream SageMaker Training pipeline, sends a high-priority Amazon SNS notification to the MLOps on-call team, and creates an incident ticket.

6. Service Comparison: Data Quality on AWS

Feature AWS Glue Data Quality PyDeequ SageMaker Model Monitor (Data Quality)

Primary Environment AWS Glue Catalog, Glue ETL, Glue Studio Apache Spark (EMR, Glue, SageMaker Processing) SageMaker Real-Time & Batch Endpoints

Rule Specification DQDL (Declarative text syntax) Python code ( pydeequ.checks.Check ) Auto-generated constraints.json from baseline

Lifecycle Stage Ingestion & Data Preparation (Domain 1) Distributed Feature Engineering (Domain 1) Post-Deployment Production Serving (Domain 4)

• Underlying Engine Managed AWS DeeQu engine Open-source DeeQu on Spark SageMaker Managed Container (DeeQu)
• Infrastructure Fully Serverless Spark cluster (EMR / Glue DPUs) Managed compute provisioned by SageMaker

Loading diagram... Automated Data Quality Validation & Quarantine Workflow