Study guide
6.1 Clause 9 Performance Evaluation
PECB ISO/IEC 42001 Lead Auditor AI Management
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Exam Orientation & AI Landscape
1.1 Current Exam Facts & PECB Credential Path 1.2 AI Concepts & ISO/IEC 22989 Terminology 1.3 AI Lifecycle & Trustworthiness Characteristics
2 Chapter 2: ISO AI Family & AIMS Purpose
2.1 ISO/IEC AI Standards Family 2.2 AIMS Purpose, PDCA & Management System Thinking 2.3 Global AI Governance Landscape for Auditors
3 Chapter 3: AIMS Requirements — Context & Leadership
3.1 Annex SL High-Level Structure & Clauses Overview 3.2 Clause 4 Context of the Organization 3.3 Clause 5 Leadership & AI Policy
4 Chapter 4: AIMS Requirements — Planning
4.1 Clause 6.1 Risks, Opportunities & AI Risk Assessment 4.2 Clause 6.1.4 AI System Impact Assessment 4.3 Clause 6.2 AI Objectives & Planning to Achieve Them 4.4 Clause 6.3 Planning of Changes
5 Chapter 5: AIMS Requirements — Support & Operation
5.1 Clause 7 Support 5.2 Clause 8 Operation & Operational Planning and Control 5.3 Clause 8 AI System Lifecycle Controls in Operation
6 Chapter 6: AIMS Requirements — Evaluation & Improvement
6.1 Clause 9 Performance Evaluation 6.2 Clause 10 Improvement 6.3 Integrated AIMS–ISMS Thinking for Auditors
7 Chapter 7: Annex A Controls — Governance & Resources
7.1 Annex A Architecture: 38 Controls Across A.2–A.10 7.2 A.2 Policies Related to AI & A.3 Internal Organization 7.3 A.4 Resources for AI Systems 7.4 A.5 Assessing Impacts of AI Systems
8 Chapter 8: Annex A Controls — Lifecycle & Data
8.1 A.6 AI System Life Cycle — Design, Development, Verification 8.2 A.6 Deployment, Operation, Monitoring & Documentation 8.3 A.7 Data for AI Systems 8.4 Bias, Fairness & Data Quality Audit Focus
9 Chapter 9: Annex A Controls — Transparency, Use & Third Parties
9.1 A.8 Information for Interested Parties 9.2 A.9 Use of AI Systems 9.3 A.10 Third-Party and Customer Relationships
10 Chapter 10: Fundamental Audit Concepts (ISO 19011)
10.1 Seven Audit Principles 10.2 Audit Types, Objectives, Criteria & Scope 10.3 Evidence-Based & Risk-Based Audit Approaches 10.4 Auditor Competence, Ethics & Impartiality
11 Chapter 11: Preparing an ISO/IEC 42001 Audit
11.1 Initiating the Audit & Feasibility 11.2 Determining Audit Objectives, Scope & Criteria 11.3 Document Review & Stage 1 Readiness 11.4 Audit Plan, Team & Working Documents
12 Chapter 12: Conducting an ISO/IEC 42001 Audit
12.1 Opening Meeting & On-Site Kickoff 12.2 Collecting Evidence: Interviews, Observation, Documents 12.3 AI-Specific Evidence: Model Cards, Lineage, Logs, Impact Assessments 12.4 Sampling Strategies & Generating Findings
13 Chapter 13: Closing an ISO/IEC 42001 Audit
13.1 Classifying Nonconformities & Observations 13.2 Audit Conclusions & Closing Meeting 13.3 Writing the Audit Report 13.4 Corrective Action Follow-Up & Verification
14 Chapter 14: Managing the AIMS Audit Program
14.1 Establishing & Implementing an Audit Program 14.2 Program Resources, Risks & Opportunities 14.3 Monitoring, Reviewing & Improving the Program 14.4 Certification Cycle: Surveillance, Recertification & Multi-Standard Audits
6.1 Clause 9 Performance Evaluation
Key Takeaways Explain This Topic AI
Clause 9.1 requires determining what to monitor and measure for the AIMS and AI systems, methods, timing, and who analyzes results—including performance, fairness, drift, and incidents where relevant.
Clause 9.2 internal audit is a first-party program that must be planned, objective, risk-based, and reported to management; it is not the same as third-party certification audit.
Clause 9.3 management review needs AIMS-specific inputs (AI objectives, risk/impact results, external issues, audit and monitoring results) and decisions on improvement, resources, and changes.
Lead auditors judge monitoring design adequacy—whether metrics can detect AI failure modes for scoped systems—not merely whether any dashboard exists.
Shared ISMS evaluation forums support an integrated AIMS only when AI metrics, AI audit coverage, and AI management-review agenda items are actually evidenced.
Last updated: August 2026
6.1 Clause 9 Performance Evaluation
Auditor focus: Clause 9 asks whether the organization knows if its AIMS and AI systems are working. “We have Grafana” is not conformity. Test what is monitored, whether methods fit AI failure modes, who acts on the data, and whether internal audit and management review close the loop to leadership decisions.
Clause 9 Performance evaluation is the Check stage of the AIMS PDCA cycle. In ISO/IEC 42001:2023 it comprises monitoring, measurement, analysis and evaluation (9.1) ; internal audit (9.2) ; and management review (9.3) . Together they answer: Is the AIMS effective? Are AI systems performing as intended and remaining trustworthy? Does top management see evidence and decide? For Lead Auditors, this is where polished policies often fail—undefined metrics, internal audit that skips high-risk models, or management review that never discusses AI objectives.
9.1 Monitoring, Measurement, Analysis and Evaluation
The organization shall determine what needs to be monitored and measured, the methods (to ensure valid results), when monitoring and measuring occur, when results are analyzed and evaluated, and who performs that work. It shall evaluate AIMS performance and effectiveness and retain documented information of the results.
What / how / when / who
Dimension Intent Weak evidence Strong evidence
What AI and AIMS topics tied to intended outcomes Only GPU/API uptime Quality, fairness/slices, drift, overrides, incidents, objective progress, control effectiveness
Methods Valid, consistent, reproducible Ad-hoc notebook checks Defined metrics, thresholds, data sources, sampling rules, versioned dashboards
When Frequency and triggers fit risk/lifecycle “When we have time” Continuous in production; periodic offline; event-driven after retrain/incident
Who Competent measurement and analysis owners Unowned Slack channel Named MLOps/product/risk owners; escalation to AIMS owner
AI-relevant metric families
Generic IT service metrics are necessary but not sufficient :
Family Examples Why auditors care
Performance / quality Accuracy, precision/recall, error rate, inference SLOs Silent degradation after data or model change
- Fairness / bias Group disparity, slice error rates Equity treatments are monitored, not only promised
- Drift / data quality Feature drift, prediction drift, train–serve skew Core production ML failure mode
- Incidents & safety AI incidents, overrides, rollbacks Links Clause 8 operation to Clause 10 improvement
AIMS process metrics % systems with current impact assessment; finding closure time Measures the management system, not only the model
Objectives Status vs Clause 6.2 AI objectives Feeds 9.3 management review
Design adequacy: Judge whether monitoring can detect foreseeable failure modes for scoped systems. A high-risk credit model with no fairness or drift monitoring is a 9.1 design gap even at 99.9% uptime. Over-instrumenting a low-risk internal tool while ignoring a customer-facing high-impact system also signals weak risk-based design.
Sampling probes: Pick 2–3 systems (prefer high risk). Request metrics, owners, thresholds, last analysis. Trace one breach: notification, decision, retained records. Map residual risks from risk/impact assessments to metrics. Check that retrain/redeploy triggers re-evaluation.
Trap: Offline development validation is not the same as ongoing operational evaluation . Clause 9.1 expects continuing evaluation over time, not a single go-live accuracy report.
9.2 Internal Audit
The organization shall conduct internal audits at planned intervals to determine whether the AIMS conforms to its own requirements and ISO/IEC 42001 and is effectively implemented and maintained . It shall maintain an audit program covering frequency, methods, responsibilities, planning, and reporting—considering process importance, change, and prior results. Auditors shall be selected to ensure objectivity and impartiality . Results go to relevant management ; documented information evidences the program and results.
Internal audit vs certification audit
Aspect Internal audit (9.2) Certification (third-party)
Party First-party program (may use contracted auditors under client control) Independent conformity assessment body
- Purpose Self-assurance; feed review and improvement Independent certification decision
- Impartiality Do not audit own work; manage conflicts CAB independence (e.g., ISO/IEC 17021-1 context)
- Frequency Risk-based planned intervals Certification cycle (initial, surveillance, recert)
- Substitution? No —certification does not cancel 9.2 No —internal audit does not replace certification
Program expectations: rolling coverage of AIMS scope and high-risk AI systems; AI-competent auditors; objectivity safeguards; findings reported to people who can resource action; open NCs linked to Clause 10.
Scenario: Annual “IT security” internal audit is claimed as AIMS coverage. Reports never sample impact assessments, model monitoring, or AI policy conformity. Treat ISMS internal audit as partial evidence of an integrated program only if AIMS criteria appear in the program and working papers. Otherwise raise 9.2: AIMS internal audit missing or ineffective.
9.3 Management Review
Top management shall review the AIMS at planned intervals for continuing suitability, adequacy and effectiveness . Reviews consider prior actions; external/internal issues; performance information (nonconformities and corrective actions, monitoring results, audit results, improvement opportunities); and resource adequacy. Outputs include decisions on continual improvement and any need for AIMS changes.
AIMS-specific inputs and outputs
- Inputs (expect AI flavor) Outputs
- Progress on AI objectives (6.2) Revise objectives, KPIs, resourcing
- AI risk and impact assessment results New treatments, controls, scope changes
External issues (regulation, model markets, customer AI requirements) Policy/process updates; interested-party communication
- 9.1 monitoring (drift, fairness, incidents) Operational changes, retrain priorities, sunset decisions
- Audit results (internal and external) CA prioritization; program adjustments
- Resource/competence gaps for AI roles Hiring, training, tooling, vendor decisions
Evidence: agenda, minutes/records, top management participation (or effective feedback loop), actions with owners and dates. Combined QMS/ISMS/AIMS reviews work only when AI items and decisions are explicit .
Scenario: Integrated management review covers security incidents and quality CAPA but never AI objectives, model performance, impact assessment status, or AI NCs—despite production clinical models in scope. Weekly data-science standups do not replace Clause 9.3. Raise 9.3 (and possibly 5.1) for failure to review AIMS with required information.
Evaluating Monitoring Design Adequacy
Scope & inventory — Any high-impact system unmonitored?
Risk & impact linkage — Residual risks mapped to metrics or justified alternatives?
Method validity — Reproducible methods; controlled evaluation data and fairness definitions?
Timeliness — Frequency vs change rate; post-incident and post-change evaluation?
Analysis & escalation — Owners, thresholds, retained results?
9.2 / 9.3 integration — Does internal audit test monitoring? Does management review see trends?
Common traps: treating certification as internal audit; security-only KPIs for AIMS; model cards without production monitoring; “AI strategy” reviews without performance data; vendor metrics with no client-side AIMS evaluation.
Clause 9 closes when measurement informs decisions . Clause 10 (next) covers nonconformity and improvement when evaluation shows gaps.