Study guide

8.4 Risk-Based Testing of ML Systems

ISTQB Certified Tester AI Testing CT-AI

Track your progress as you read

Finished sections get a check mark.

1 Introduction & Exam Overview

1.1 Current Exam Facts (CT-AI v2.0) 1.2 Scoring, K-levels & Chapter Weights

2 Introduction to AI: Systems and Capabilities

2.1 AI-Based vs Conventional Systems 2.2 Narrow AI, General AI and Super AI 2.3 Types of AI Technologies 2.4 Generative AI

3 AI Hardware, Hosting, Frameworks and Governance

3.1 Hardware for Machine Learning Systems 3.2 Development and Hosting of AI Models 3.3 Machine Learning Development Frameworks 3.4 Regulations and Standards for AI

4 Quality Characteristics and Acceptance Criteria

4.1 ISO/IEC 25059 AI Quality Characteristics 4.2 AI and Safety 4.3 Acceptance Criteria for AI-Based Systems

5 Machine Learning Forms, Workflow and Data Preparation

5.1 Forms of Machine Learning 5.2 The Machine Learning Workflow 5.3 Pretrained Models, Fine-Tuning and RAG 5.4 Data Preparation Activities

6 Datasets, Classification Metrics and Neural Networks

6.1 Training, Validation and Test Datasets 6.2 Calculating Classification Metrics 6.3 Structure of Deep Neural Networks 6.4 Neural Network Coverage Measures

7 Testing Challenges for AI-Based Systems

7.1 Locked vs Adaptive AI-Based Systems 7.2 Why Statistical Testing Is Needed 7.3 Test Oracles for AI-Based Systems

8 GenAI Testing, Test Levels and Risk-Based Testing

8.1 Testing Generative AI 8.2 Red Teaming for GenAI 8.3 Test Levels for Machine Learning Systems 8.4 Risk-Based Testing of ML Systems

9 Input Data Risks, Bias and Pipelines

9.1 Input Data Risks and Mitigations 9.2 Testing for Bias 9.3 Data Pipeline Testing

10 Representativeness, Constraints and Labels

10.1 Testing for Data Representativeness 10.2 Dataset Constraint Testing 10.3 Label Correctness Testing

11 Model Risks, Documentation and Performance Testing

11.1 ML Model Risks and Mitigations 11.2 ML Model Documentation and Review 11.3 Functional Performance Testing of Probabilistic MLS

12 Adversarial, Metamorphic and Drift Testing

12.1 Adversarial Testing 12.2 Metamorphic Testing 12.3 Drift Testing

13 Overfitting, A/B Testing and Back-to-Back Testing

13.1 Testing for Overfitting and Underfitting 13.2 A/B Testing for ML Systems 13.3 Back-to-Back Testing

14 ML Development and Deployment Testing

14.1 ML Development Risks and Mitigations 14.2 ML System Deployment Testing

8.4 Risk-Based Testing of ML Systems

Key Takeaways Explain This Topic AI

Risk-based testing applies to all systems; most AI regulations require a risk-based approach, and AI-based systems pose unique risks.

There is no standard ML risk taxonomy; one useful split follows the ML workflow: development project risks, input-data product risks, and model product risks.

Development and project risks such as algorithm selection, evaluation approach, and framework security are detailed in Chapter 7.

Input-data product risks such as bias, pipeline defects, and unrepresentative data are detailed in Chapter 5; model product risks such as missed metrics, overfitting, and adversarial examples are detailed in Chapter 6.

A second split uses ISO/IEC 25059 quality characteristics; specialized MLS techniques include data pipeline testing, adversarial testing, and algorithm or model suitability review.

Last updated: September 2026

Risk-based testing is the default, not an AI extra

Risk-based testing applies to all systems , with or without AI. You already prioritize tests by likelihood and impact. CT-AI v2.0 insists on that discipline because most regulatory frameworks , published or in development, require a risk-based approach to developing and managing AI-based systems. The EU AI Act is one example of that expectation; it is not the only one. Learning objective AI-4.3.2 is K2 : explain how risk-based testing is applied to machine learning systems.

AI-based systems pose unique risks , so testing them differs from testing non-AI systems. A conventional defect might crash a page. An MLS defect might silently deny credit, invent medical advice, or leak a training record through a prompt. Impact is often statistical, delayed, or distributed across a population, which is why a purely scripted happy path is not a strategy.

Unique does not mean mysterious. It means the failure modes travel through data, induced behavior, and probabilistic outputs, so the test strategy must name those paths instead of copying a CRUD application's risk list and stopping there.

There is no standard ML risk taxonomy

There is no standard way to categorize ML risks . That sentence is itself an exam point. Do not invent a fake ISO number for the ML risk list. The syllabus then offers two useful splits you are expected to explain.

Split 1 — follow the ML workflow

ML-specific risks attach to development of the MLS (project risks) and to the MLS itself (product risks) . One way to categorize them is to divide the ML workflow into three areas.

Development (project risks) — concerned with the ML algorithm , development of the model , and the ML development framework . Example project risks include sub-optimal algorithm selection , poor selection of evaluation approach , and framework security vulnerabilities . See Chapter 7 . Testers contribute by reviewing whether the team picked a model class the data can support, whether the evaluation set is independent, and whether notebook dependencies are pinned and scanned. A beautiful dashboard that reports accuracy on the training set is an evaluation-approach risk, not a user-interface risk.

Input data (product risks) — concerned with providing training data to support ML and production data used by the model in its operational environment. Example product risks include biased training data , data-pipeline defects , and unrepresentative training data . See Chapter 5 . A pipeline that drops a rare but high-impact class is a product risk even if the algorithm choice was sound. Unrepresentative data is not cured by a more fashionable architecture.

Model (product risks) — concerned with the generated ML model . Example product risks include failure to achieve required ML functional performance measures , an overfitted model , and susceptibility to adversarial examples . See Chapter 6 . Overfitting is a model product risk even when the pipeline and the framework were fine.

Memorize the mapping for the exam: development and project → Chapter 7; input-data product → Chapter 5; model product → Chapter 6. Items love to swap those destinations.

Split 2 — ISO/IEC 25059 quality characteristics

Another way to categorize risks associated with AI is according to the quality characteristics defined in ISO/IEC 25059 , which extends ISO/IEC 25010 for AI systems. From a testing perspective those characteristics shape objectives and acceptance criteria. AI-specific characteristics you met in Chapter 2 include AI functional correctness , functional adaptability , user controllability , transparency , AI robustness , intervenability , and societal and ethical risk mitigation .

A risk register can be a matrix of workflow area versus quality characteristic. Unrepresentative training data (Chapter 5) threatens fairness under societal and ethical risk mitigation and also threatens AI functional correctness for a minority slice. Weak intervenability is a product risk even when recall looks strong: if no operator can stop a harmful action in time, the quality characteristic has failed.

Loading diagram... Two ways to split ML risks for test strategy

Specialized techniques that exist because the risks are ML-specific

Several forms of testing address these risks and are designed for MLS . The syllabus names, as instances:

  • Data pipeline testing
  • Adversarial testing
  • Review of algorithm/model suitability

Chapters 5, 6, and 7 cover several of these in depth. Your job at AI-4.3.2 is to explain how risk-based testing is applied : pick a split, name example risks, and name a technique that mitigates them. You are not asked here to calculate an adversarial perturbation or to run a full pipeline suite.

Use the techniques where the register says impact is high. Data pipeline testing answers input-data product risk. Adversarial testing answers model product risk of susceptibility to adversarial examples. Algorithm and model suitability review answers development project risk of sub-optimal algorithm selection. Mixing those up on the exam is a common fail.

Worked risk register for an imaging MLS

A hospital deploys a locked imaging model that flags likely fractures on X-rays, with a radiologist still issuing the report. The strategy meeting lasts forty minutes and produces the table below — that is risk-based testing applied, not a slogan.

Risk Split Example test response

Team picks a tiny CNN because the framework tutorial used it Development / project (Chapter 7) Algorithm and model suitability review against image resolution and class imbalance

Training studies come from one scanner vendor Input data product (Chapter 5) Representativeness testing; pipeline tests for vendor-tag handling

Nightly DICOM feed drops a required metadata field Input data product (Chapter 5) Data pipeline testing in a production-like integration environment

Hold-out recall looks strong; live recall drops on pediatric films Model product (Chapter 6) Additional model testing on the under-served slice; ML functional performance monitoring

Stickers on the cassette fool the detector Model product (Chapter 6) Adversarial testing of input perturbations

Quantized edge model on the clinic PC disagrees with the server model Conventional system test, triggered by model-change risk Confirmation testing after compression (see 8.3)

A GenAI report-drafting add-on could invent findings Product risk across safety and misinformation Red teaming (4.2.2) plus acceptance criteria on AI functional correctness

Notice the pattern: unique AI risks do not replace component or system tests; they prioritize extra ML levels and specialized techniques where impact is high. A low-risk internal clustering demo may skip heavy adversarial testing. A high-risk clinical or credit system should not.

When regulators ask for a risk-based approach, they are asking to see this kind of register, the tests it drove, and the residual risk you accepted — not a claim that the team ran accuracy. CT-AI v2.0 expects you to explain that application, using either the workflow split or the ISO/IEC 25059 split, and to name the specialized techniques that belong with those risks.