Study guide
7.3 Choosing an Effective Assessment Method and Criterion-Referenced Testing
FAA Fundamentals of Instructing FOI
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: Risk Management, ADM, and Single-Pilot Resource Management
1.1 Principles of Aeronautical Decision-Making (ADM) & the DECIDE Model 1.2 Single-Pilot Resource Management (SRM) & Risk Assessment Models 1.3 Teaching Practical Risk Management & Scenario-Based Training
2 Chapter 2: Human Behavior and the Psychology of Learning
2.1 Human Nature, Motivation, and Learner Needs 2.2 Defense Mechanisms and Emotional Barriers 2.3 Adult Learning Principles and Learner Differences
3 Chapter 3: Learning Theory, Laws of Learning, and Learning Domains
3.1 Definitions of Learning and Perception 3.2 Thorndike's Laws of Learning (REEPIR) 3.3 Domains and Levels of Learning 3.4 Learning Theories, Knowledge Acquisition, and Learning Styles
4 Chapter 4: Skill Acquisition, Memory, and Transfer of Learning
4.1 Stages of Skill Acquisition and Types of Practice 4.2 Memory Systems, Forgetting, and Transfer of Learning 4.3 Retention of Learning, Task Management, and Error Management
5 Chapter 5: Effective Communication in Aviation Training
5.1 The Communication Model and Process 5.2 Barriers to Effective Communication 5.3 Developing Instructional Communication and Listening Skills
6 Chapter 6: The Teaching Process and Delivery Methods
6.1 The Four-Step Teaching Process 6.2 Traditional Teaching Delivery Methods 6.3 Demonstration-Performance and Problem-Based Learning 6.4 Instructional Aids, Training Media, Drill and Practice, and Test Preparation Material
7 Chapter 7: Assessment Theory and Traditional Written Testing
7.1 Purposes and Characteristics of Effective Assessment 7.2 Traditional Written Test Design and Evaluation Metrics 7.3 Choosing an Effective Assessment Method and Criterion-Referenced Testing
8 Chapter 8: Authentic Assessment, Questioning, and Critique
8.1 Authentic Assessment and Collaborative Evaluation 8.2 Effective Questioning and Instructional Critique Methods
9 Chapter 9: Planning Instructional Activity and Lesson Design
9.1 Training Courses, Syllabi, and Curriculum Structure 9.2 Learning Objectives and Lesson Plan Construction
10 Chapter 10: Instructor Responsibilities and Professionalism
10.1 Core Aviation Instructor Responsibilities 10.2 Flight Instructor Specific Duties and Professionalism
11 Chapter 11: Techniques of Flight Instruction
11.1 Learning Obstacles and Flight Cockpit Dynamics 11.2 Core Flight Instruction Techniques and Cockpit Procedures 11.3 Integrating Ground, Simulation, and Flight Instruction, and Assessing Piloting Ability
7.3 Choosing an Effective Assessment Method and Criterion-Referenced Testing
Key Takeaways Explain This Topic AI
FAA-H-8083-9B prescribes a four-step process for choosing an assessment: determine level-of-learning objectives, list indicators of desired behaviors, establish criterion objectives, and develop criterion-referenced test items.
The development process for criterion-referenced tests follows a general-to-specific pattern and applies to the cognitive and affective domains as well as psychomotor skill testing.
Authentic assessment may be less useful than traditional assessment early in training because the learner does not yet have enough knowledge to participate fully; full participation becomes possible at the application and correlation levels.
A criterion objective adds the conditions under which the behavior is performed and the criteria that must be met; if performance-based objectives were written into the lesson plans, the criterion objectives already exist.
Practical tests defined in the ACS/PTS are criterion-referenced because the objective is for every successful applicant to meet the same required standard of knowledge, skill, and safety, so the instructor should evaluate all tasks before recommending an applicant.
Last updated: September 2026
7.3 Choosing an Effective Assessment Method and Criterion-Referenced Testing
Quick Answer: When deciding how to assess learner progress, FAA-H-8083-9B gives aviation instructors a four-step process : (1) determine level-of-learning objectives , (2) list indicators/samples of desired behaviors , (3) establish criterion objectives , and (4) develop criterion-referenced assessment items . The process runs general to specific and works for the cognitive and affective domains as well as psychomotor skill testing. A criterion objective adds conditions and criteria to the expected behavior — so if you wrote performance-based objectives into your lesson plans, your criterion objectives already exist. The ACS and PTS are criterion-referenced standards, which is why the instructor evaluates every task before recommending an applicant.
Why Instructors Get This Backward
The common failure is building the test first: an instructor writes twenty questions about carburetor icing, gives the quiz, and only afterward asks what the score was supposed to prove. The handbook's process runs the other direction — general to specific — and each step constrains the next. Start with what level of learning you intended to produce, and the test items fall out of it.
There is also a timing decision hidden in this. The handbook cautions that authentic assessment may not be as useful as traditional assessment in the early phases of training , because the learner "does not have enough information about the concepts or knowledge to participate fully." This tracks the knowledge-acquisition sequence from section 3.4: a learner exposed to a new topic first acquires and memorizes facts, then organizes them into understanding. Only once the learner possesses enough knowledge to analyze, synthesize, and evaluate — the application and correlation levels — can they participate fully in a collaborative, learner-centered assessment. Handing a first-lesson student a self-assessment worksheet is not learner-centered; it is abandonment.
Step 1: Determine Level-of-Learning Objectives
State the individual objectives as general, level-of-learning objectives . The objective should measure one of the learning levels in the cognitive, affective, or psychomotor domains. The cognitive levels named by the handbook are knowledge, comprehension, application, analysis, synthesis, and evaluation .
The handbook's worked example is a maintenance one, and it is worth following exactly because the FOI tests the boundaries between levels:
Understanding-level objective: "Describe how to perform a compression test on an aircraft reciprocating engine."
Notice what this objective does not require:
- Task Level it would actually require
- Describe how to perform the compression test Understanding — the stated objective
- Actually perform a compression test Application
- Compare results of compression tests on different engines Application
- Design a compression test for a different type of engine Correlation
- Interpret the results of the compression test Correlation
The general level-of-learning objective is "a good starting point for developing a test because it defines the scope of the learning task ." Get the level wrong and every later step measures the wrong thing — which is exactly how an instructor ends up asking a pre-solo student to evaluate a crosswind technique they have only been taught to describe .
Step 2: List Indicators / Samples of Desired Behaviors
List the behaviors that give the best indication that the objective has been achieved . The instructor selects behaviors that (a) can be measured and (b) give the best evidence of learning.
Continuing the compression-test example, if the learner is expected to display the understanding level, the test answers should describe:
- appropriate tools and equipment ;
- the proper equipment setup ;
- appropriate safety procedures ; and
the steps used to obtain compression readings.
Two handbook constraints govern this step:
The overall test must be comprehensive enough to give a true representation of the learning to be measured .
It is not usually feasible to measure every aspect of a level-of-learning objective — so by carefully choosing samples of behavior, the instructor obtains adequate evidence of learning.
That pair of statements is the resolution to a classic FOI tension: comprehensiveness is a required characteristic of a good test, yet no test can cover everything. Sampling deliberately is the answer; sampling accidentally is not.
Step 3: Establish Criterion Objectives
A criterion objective is a performance-based objective. In addition to the behavior expected, it states:
the conditions under which the behavior is to be performed, and
the criteria that must be met.
The handbook makes the labor-saving point explicit: if the instructor developed performance-based objectives while creating lesson plans, the criterion objectives have already been formulated. The criterion objective then "provides the framework for developing the test items used to measure the level of learning objectives."
Handbook example: "The learner will demonstrate understanding of compression test procedures for reciprocating aircraft engines by completing a quiz with a minimum passing score of 70 percent."
Break that apart against the three-element structure taught in section 9.2:
- Element In the example
- Behavior / skill Demonstrate understanding of compression test procedures for reciprocating aircraft engines
- Conditions By completing a quiz
- Criteria / standard Minimum passing score of 70 percent
- Step 4: Develop Criterion-Referenced Assessment Items
Write items that measure the behaviors described in the criterion objective . In the compression-test example, the questions should cover every area necessary to give evidence of understanding the procedure. The diagnostic payoff comes afterward: the questions missed identify the areas that were not adequately covered — the test grades the instruction as well as the learner.
The handbook adds two practical applications:
A pre-solo knowledge test measures whether the learner understands the regulations, the local area, the aircraft type, and the procedures to be used, so it must be written against those objectives. Individual instructors should develop their own tests to measure their own learners' progress.
A readiness test for an FAA knowledge test should be based on the objectives of all the lessons the learner has received, not just the recent ones.
Performance Testing and the ACS/PTS as Criterion-Referenced Standards
Aviation training also involves performance tests for maneuvers or procedures. The flight instructor does not administer the practical test for a pilot certificate, and the maintenance instructor does not administer the AMT oral and practical exam — but instructors do exactly the same kind of skill or performance testing in training. Performance testing is desirable for evaluating training that involves an operation, a procedure, or a process , and the instructor's job is to prepare the learner for those tests.
Practical tests for pilots and maintenance technicians are criterion-referenced tests : the objective is for all successful applicants to meet the high standards of knowledge, skill, and safety required by the regulations — not to rank applicants against each other. The purpose of the ACS/PTS is to delineate the standards by which FAA inspectors, designated pilot examiners (DPEs) , and designated maintenance examiners (DMEs) conduct tests, reflecting 14 CFR parts 61, 65, 91, the AIM, and pertinent advisory circulars and handbooks.
ACS/PTS structure Definition
Areas of operation Phases of the practical test arranged in a logical sequence within each standard, usually beginning with preflight preparation and ending with postflight procedures
Tasks Titles of knowledge areas, flight procedures, or maneuvers appropriate to an area of operation, including references to the applicable regulations or publications
Private pilot applicants are evaluated in all tasks of each area of operation.
Flight instructor applicants are evaluated on one or more tasks in each area of operation, with certain required tasks identified by notes immediately following the area of operation titles.
Because an evaluator may cover every task , the handbook instructs that the instructor evaluate all of the tasks before recommending a pilot or maintenance applicant for the practical test. That evaluation is "not necessarily formal," but it should adhere to criterion-referenced testing — measured against the published standard, not against how the learner compares to last month's student.
Putting It Together: A Worked Flight-Training Example
Lesson: short-field takeoff and maximum performance climb, private pilot level.
Step Applied to this lesson
1. Level-of-learning objective Application level: the learner will perform a short-field takeoff and maximum performance climb. (Note what this excludes — describing the technique would be understanding; deciding whether a given strip is usable on a 92°F day would be correlation.)
2. Indicators of desired behavior Correct flap setting for the POH; brakes held while power is set; rotation at the POH speed; V X V_X V X held to clear the obstacle; transition to V Y V_Y V Y ; flap retraction at a safe altitude; checklist completion
3. Criterion objective "Given a short-field departure from a 2,400-foot runway with a 50-foot obstacle, the learner will perform a short-field takeoff, establishing and maintaining V X V_X V X ± 5 knots until the obstacle is cleared, then transitioning to V Y V_Y V Y ."
4. Criterion-referenced items An oral item on POH flap and speed selection; a written item on the difference between V X V_X V X and V Y V_Y V Y ; an in-flight performance item scored against the ±5-knot tolerance. Items missed tell the instructor which part of the lesson failed
FOI Exam Traps & Common Misconceptions
Trap 1 (order of the four steps): Level-of-learning objectives → indicators of desired behavior → criterion objectives → criterion-referenced items. The pattern is general to specific ; any option that starts with writing test questions is reversed.
Trap 2 (criterion-referenced vs. norm-referenced): Criterion-referenced means measured against a published standard , with the goal that all successful applicants meet it. If an option describes ranking learners against each other or grading on a curve, it is describing norm-referenced testing and is wrong for the ACS/PTS.
Trap 3 (when authentic assessment works): Authentic and collaborative assessment are less useful in the early phases of training ; the learner can participate fully once they reach the application and correlation levels.
Trap 4 (you already wrote the criterion objectives): If the stem mentions that performance-based objectives exist in the lesson plans, the keyed answer is that the criterion objectives have already been formulated .
Trap 5 (who evaluates which tasks): Private pilot applicants are evaluated in all tasks of each area of operation; flight instructor applicants are evaluated on one or more tasks in each area of operation. Either way, the recommending instructor should evaluate all tasks first, because the evaluator may cover every one.