Study guide
10.2 Dataset Constraint Testing
ISTQB Certified Tester AI Testing CT-AI
Track your progress as you read
Finished sections get a check mark.
1 Introduction & Exam Overview
1.1 Current Exam Facts (CT-AI v2.0) 1.2 Scoring, K-levels & Chapter Weights
2 Introduction to AI: Systems and Capabilities
2.1 AI-Based vs Conventional Systems 2.2 Narrow AI, General AI and Super AI 2.3 Types of AI Technologies 2.4 Generative AI
3 AI Hardware, Hosting, Frameworks and Governance
3.1 Hardware for Machine Learning Systems 3.2 Development and Hosting of AI Models 3.3 Machine Learning Development Frameworks 3.4 Regulations and Standards for AI
4 Quality Characteristics and Acceptance Criteria
4.1 ISO/IEC 25059 AI Quality Characteristics 4.2 AI and Safety 4.3 Acceptance Criteria for AI-Based Systems
5 Machine Learning Forms, Workflow and Data Preparation
5.1 Forms of Machine Learning 5.2 The Machine Learning Workflow 5.3 Pretrained Models, Fine-Tuning and RAG 5.4 Data Preparation Activities
6 Datasets, Classification Metrics and Neural Networks
6.1 Training, Validation and Test Datasets 6.2 Calculating Classification Metrics 6.3 Structure of Deep Neural Networks 6.4 Neural Network Coverage Measures
7 Testing Challenges for AI-Based Systems
7.1 Locked vs Adaptive AI-Based Systems 7.2 Why Statistical Testing Is Needed 7.3 Test Oracles for AI-Based Systems
8 GenAI Testing, Test Levels and Risk-Based Testing
8.1 Testing Generative AI 8.2 Red Teaming for GenAI 8.3 Test Levels for Machine Learning Systems 8.4 Risk-Based Testing of ML Systems
9 Input Data Risks, Bias and Pipelines
9.1 Input Data Risks and Mitigations 9.2 Testing for Bias 9.3 Data Pipeline Testing
10 Representativeness, Constraints and Labels
10.1 Testing for Data Representativeness 10.2 Dataset Constraint Testing 10.3 Label Correctness Testing
11 Model Risks, Documentation and Performance Testing
11.1 ML Model Risks and Mitigations 11.2 ML Model Documentation and Review 11.3 Functional Performance Testing of Probabilistic MLS
12 Adversarial, Metamorphic and Drift Testing
12.1 Adversarial Testing 12.2 Metamorphic Testing 12.3 Drift Testing
13 Overfitting, A/B Testing and Back-to-Back Testing
13.1 Testing for Overfitting and Underfitting 13.2 A/B Testing for ML Systems 13.3 Back-to-Back Testing
14 ML Development and Deployment Testing
14.1 ML Development Risks and Mitigations 14.2 ML System Deployment Testing
10.2 Dataset Constraint Testing
Key Takeaways Explain This Topic AI
Dataset constraint testing checks whether ML data satisfy a predefined rule set that plays the same role as a schema: single-value, multi-value, and comparison rules.
Single-value constraints include Missing, Range, and Type (an integer attribute must not arrive as a string or a real).
Multi-value constraints include Sum (Formula 1 points cannot exceed 102 and must exceed 50.5), Count of non-null values, Duplicate (often zero allowed), Useful (all-unique IDs or timestamps give no learnable pattern), and Outlier.
Comparison constraints include Greater Than (lines of code must exceed defective lines of code) and Correlate (marks at least 1.33 standard deviations above the mean also carry grade A).
Automate the checks in the pipeline at ML scale; send training anomalies to data scientists and operational failures to operations staff.
Last updated: September 2026
Constraints as a schema for ML data
Dataset constraint testing asks whether the data adhere to predefined rules . Think of the rule set as a schema for ML data . A database schema names types, nullability, and relationships. An ML constraint set does the same job for files that may never live in a relational table: it is a logical model of what correct data means. If a constraint fails, the data are not ready to train on, or not safe to score in production, even if a scatter plot looks tidy.
Manual spot checks do not scale to the size of modern training dumps or live feature streams. Automate constraint tests in the data pipeline . Route reports to data scientists when training data look wrong, and to operations when live inputs look wrong. The same engine can serve both audiences; the difference is who owns the follow-up.
Single-value constraints
A single-value constraint inspects one attribute on one instance.
Missing. The cell is empty, null, or the attribute is absent from the record. A delay classifier that requires weight_kg must fail a row where weight is blank. Missingness can also mean the column never arrived from an upstream join.
Range. The value must lie in a stated interval. Parcel weight 0.05 kg to 30 kg is a domain range; an 85 kg "parcel" is not a legal input for that product, whether or not it is also a statistical outlier.
Type. The value must match the declared type. If n_stops is specified as an integer, the string "3" or the real 3.7 is a type defect. Integers, strings, and reals are not interchangeable just because a parser can coerce them later.
Multi-value constraints
A multi-value constraint looks across several values, usually one attribute over many instances.
Sum. The total of a column must equal, exceed, or not exceed a limit. The standard teaching example is Formula 1 race points: the total awarded cannot exceed 102 and must exceed 50.5 . Worked numbers: a completed race file lists 25, 18, 15, 12, 10, 8, 6, 4, 2, 1, and 1 fastest-lap point. Sum = 102, which satisfies both bounds. If a second 25 is appended (two "winners"), sum = 127, which exceeds 102 and the Sum constraint fires. If a truncated file sums to 40, the total does not exceed 50.5, and Sum fires on the lower bound. Testers apply the same pattern to ML files: daily bonus points in a courier league, allocated minutes in a shift roster, or grouped probabilities that must total near 1.0.
Count. The number of non-null values for an attribute or for instances must equal, exceed, or not exceed a threshold. A batch that requires at least five labelled delayed values fails if only three labels arrived.
Duplicate. Identical or near-identical attribute values or whole instances are limited, often to zero . Two rows with parcel_id P-102 is a Duplicate failure when identifiers must be unique.
Useful. An attribute used as a learning feature should show some repeated values . If every entry is unique, like a raw ID or a high-resolution timestamp, the column typically provides no useful pattern for the model to learn. Duplicate and Useful are easy to mix up on a two-point item. Duplicate asks whether the same ID appeared twice as a data defect . Useful asks whether, even if IDs are perfectly unique, you should feed that column to the model . Unique IDs can pass Duplicate (limit zero duplicates) and still fail Useful as features.
Outlier. Values that sit far from the rest of the distribution are flagged. A domain range check and an outlier check can both fire on the same cell: 85 kg may be outside 0.05–30 and also a statistical outlier in a batch of 1–5 kg parcels.
Comparison constraints
Greater Than. One attribute must exceed another on the same instance. The teaching example: lines of code (LOC) for a program must exceed its count of defective LOC . Module A with loc=500 and defective_loc=40 passes. Module B with loc=80 and defective_loc=80 fails because 80 is not greater than 80. Module C with loc=40 and defective_loc=55 fails as well. The same pattern appears as distance_km must exceed last_mile_km .
Correlate. Values of one attribute must travel with another. The teaching example: every student whose marks sit at least 1.33 standard deviations above the mean also has grade A .
Worked marks: 62, 68, 70, 71, 94. Sum = 365, mean = 73. Deviations: -11, -5, -3, -2, +21. Squared deviations: 121, 25, 9, 4, 441. Sum of squares = 600. Sample standard deviation s = sqrt(600 / 4) = sqrt(150) ≈ 12.25. Then 1.33 × s ≈ 16.29. Threshold = 73 + 16.29 ≈ 89.29. Only the student with 94 clears the threshold. If that student is labelled B, Correlate fires . If labelled A, Correlate is satisfied for this batch. Testers compute the threshold from the dataset under test (or from a documented reference rule) and then check the paired attribute. Do not eyeball "94 looks high" and skip the 1.33 calculation; the exam item is testing that you can apply the rule.
Loading diagram... Three families of dataset constraints
Worked scenario: which constraints fire?
This is the shape of a two-point apply item : you are given a mini-table and a constraint list, and you must mark every rule that fires, not just the first defect you notice.
A pipeline for a parcel-delay classifier declares these rules for a six-row batch:
Missing — weight_kg is required.
Range — weight_kg in [0.05, 30].
Type — n_stops is an integer.
Sum — total bonus_points must exceed 50.5 and must not exceed 102.
Count — at least five non-null delayed labels.
Duplicate — parcel_id unique (zero duplicates allowed).
Useful — candidate model features should not be all-unique IDs or timestamps.
Outlier — extreme weight_kg relative to the batch.
Greater Than — distance_km exceeds last_mile_km .
Correlate — any row with priority_score at least 1.33 standard deviations above the batch mean has service_tier = express.
- parcel_id weight_kg distance_km last_mile_km n_stops bonus_points delayed scan_ts service_tier priority_score
- P-101 2.4 45 8 3 18 0 08:00 standard 50
- P-102 1.1 12 12 1 25 0 08:05 express 88
- P-103 (blank) 90 20 4 12 1 08:10 standard 42
- P-102 4.0 30 5 2 20 0 08:12 standard 55
- P-105 2.8 40 6 "3" 15 0 08:15 standard 48
- P-106 85.0 25 4 2 22 1 08:20 standard 91
Walk every rule, in order:
Missing fires on P-103 (blank weight).
Range fires on P-106 (85.0 is above 30).
Type fires on P-105 ( n_stops is the string "3" , not an integer).
Sum: 18 + 25 + 12 + 20 + 15 + 22 = 112 , which exceeds 102, so Sum fires. The lower bound 50.5 is met; the upper bound is not.
Count does not fire: six delayed labels are present, and the rule asked for at least five.
Duplicate fires: P-102 appears on two rows.
Useful fails for parcel_id and scan_ts as features (identifiers and timestamps with no repeated learning pattern). Duplicate can fail on the repeated P-102 at the same time; the two rules still answer different questions.
Outlier fires on 85.0 kg beside a cluster of roughly 1–4 kg weights.
Greater Than fires on the first P-102 row: distance 12 is not greater than last_mile 12.
Correlate: scores 50, 88, 42, 55, 48, 91. Mean = 374 / 6 ≈ 62.33. Sample standard deviation ≈ 21.47. Threshold ≈ 62.33 + 1.33 × 21.47 ≈ 90.9. Only 91 meets the threshold. That row is standard , not express , so Correlate fires.
Exam tactic: list every constraint, mark fire / no-fire, and only then choose the option that matches the full set. Two-point items often hide one clean pass (here, Count) next to several failures so that "everything is broken" is also wrong. Stopping after the blank weight would miss Sum, Duplicate, Type, Greater Than, and Correlate — and would miss most of the marks.
When the same checks run in production, the pipeline should emit a machine-readable report: which rule, which rows, which split (training versus live). Data scientists use training failures to stop a fit. Operations use live failures to quarantine inputs, open an incident, or trigger a fallback policy. Do not treat a red constraint dashboard as a model-accuracy problem until the data rules have been examined.