Study guide
8.1 Iterators and the Iteration Protocol
OpenEDG PCAP — Certified Associate in Python Programming Exam
Track your progress as you read
Finished sections get a check mark.
1 Chapter 1: The PCAP-31-03 Exam Blueprint and Test-Day Logistics
1.1 The PCAP-31-03 Blueprint, Scoring, and Exam-Day Logistics
2 Chapter 2: Modules, Packages, and the Standard Library
2.1 Module Imports and Namespace Mechanics 2.2 Module Construction and Execution 2.3 Package Hierarchy and Navigation 2.4 Standard Library Modules and Pip
3 Chapter 3: Exceptions and Error Handling
3.1 Exception Control Flow and Syntax 3.2 The Built-in Exception Hierarchy 3.3 Raising Exceptions and Assertions 3.4 Custom Exceptions and Exception Chaining
4 Chapter 4: Strings, Encodings, and Text Processing
4.1 Character Encodings and String Immutability 4.2 String Indexing, Slicing, and Comparison 4.3 String Inspection and Classification Methods 4.4 String Transformation and Formatting Methods 4.5 Iterating, Sorting, and Comparing Strings Against Other Types
5 Chapter 5: Object-Oriented Programming Fundamentals
5.1 Classes, Instances, and Attributes 5.2 Encapsulation, Privacy, and Name Mangling 5.3 Dunder Methods and Object Lifecycle 5.4 Discovering Class Structure: __name__, __module__, __bases__, and Identity
6 Chapter 6: Advanced OOP, Inheritance, and Metaprogramming
6.1 Single Inheritance, Method Overriding, and Polymorphism 6.2 Multiple Inheritance and Method Resolution Order (MRO) 6.3 Class Methods, Static Methods, and Properties
7 Chapter 7: Functional Python, Comprehensions, and Closures
7.1 List, Dictionary, and Set Comprehensions 7.2 Lambda Expressions and Higher-Order Functions 7.3 Variable Scopes, the LEGB Rule, and Closures
8 Chapter 8: Iterators, Generators, and File I/O
8.1 Iterators and the Iteration Protocol 8.2 Generator Functions and Generator Expressions 8.3 File Input/Output and Context Managers 8.4 Predefined Streams, Handles vs. Streams, and errno Diagnostics
8.1 Iterators and the Iteration Protocol
Key Takeaways Explain This Topic AI
An iterable is any object capable of returning an iterator via iter () or supporting index-based access via getitem (), while an iterator is a stateful stream producing values via next ().
The Python Iterator Protocol requires an iterator to implement both next () (to compute values or raise StopIteration) and iter () (returning self so the iterator can be looped over).
The built-in iter() function retrieves an iterator from an iterable, or creates a callable-sentinel iterator when called with two arguments: iter(callable, sentinel).
The next(iterator[, default]) function advances the iterator by one step, returning the next value or a provided default fallback value when StopIteration is raised.
Iterators are strictly single-pass streams; once exhausted, they cannot be reset or rewound, and all subsequent next() calls will continue raising StopIteration.
Last updated: August 2026
Iterators and the Iteration Protocol
Iteration is one of the foundational paradigms of Python programming. From traversing elements in a list to streaming gigabytes of log entries line by line, Python uses a unified, standard mechanism known as the Iteration Protocol . Understanding the internal mechanics of this protocol—how Python distinguishes between containers that hold data and objects that traverse data—is a core requirement for the PCAP certification.
1. Iterables vs. Iterators: The Fundamental Distinction
Many Python programmers use the terms iterable and iterator interchangeably, but in Python's object model, they represent two distinct concepts with completely different responsibilities.
What is an Iterable?
An iterable is an object capable of returning its members one at a time. It serves as a data container or sequence representation. Examples of built-in iterables include:
- Sequences: list , tuple , str , bytes , range
- Non-sequence collections: dict , set , frozenset
- File streams and generator objects
An object is an iterable if it satisfies either of two conditions:
It implements the __iter__() method, which returns an iterator object .
It implements the legacy sequence protocol method __getitem__() accepting integer indices starting from 0 and raising IndexError when out of bounds.
What is an Iterator?
An iterator is a stateful stream object that produces the next value in a sequence upon demand. It maintains internal state pointing to the current position within the iteration traversal.
An object is an iterator if and only if it implements the Iterator Protocol :
__next__() : Returns the next element in the stream, or raises the built-in StopIteration exception when no more elements remain.
__iter__() : Returns the iterator object itself ( return self ).
Concrete Comparison
- Attribute / Behavior Iterable (e.g., list , str ) Iterator (e.g., list_iterator )
- Primary Role Holds or represents data Tracks traversal state across data
- Required Methods __iter__() (or __getitem__() ) __iter__() AND __next__()
- Stateful Traversal Stateless container Stateful (consumed as traversed)
- Multiple Passes Reusable (can iterate many times) Single-pass (exhausted after traversal)
Direct next() Call Raises TypeError ( 'list' object is not an iterator ) Returns next item or raises StopIteration # A list is an iterable, NOT an iterator
numbers = [10, 20, 30]
try:
next(numbers)
except TypeError as e:
print(e) # 'list' object is not an iterator
- # Calling iter() on the iterable produces an iterator
- num_iter = iter(numbers)
- print(type(num_iter)) # <class 'list_iterator'>
- print(next(num_iter)) # 10
- print(next(num_iter)) # 20
- print(next(num_iter)) # 30
try:
next(num_iter)
except StopIteration:
- print("Iterator is fully exhausted!")
- 2. The Iterable and Iterator Protocols
Python's iteration mechanics rely on object-oriented special methods (dunder methods).
The Iterable Protocol: __iter__()
When Python needs to iterate over an object obj (for example, at the start of a for loop or when calling list(obj) ), it invokes obj.__iter__() . This method must return an iterator instance. class NumberCollection:
def __init__(self, data):
self.data = data
def __iter__(self):
- # Return a new iterator over self.data
- return iter(self.data)
- The Iterator Protocol: __next__() and __iter__()
To qualify as an iterator, a class must implement two methods:
__next__(self) : Advances the iterator state and returns the next item. When the stream is exhausted, it must raise StopIteration .
__iter__(self) : Returns self . This allows an iterator to be passed directly into loops, list constructors, and any function expecting an iterable.
Important
Why Iterators Must Implement __iter__() Returning self : In Python, every iterator is also an iterable because it implements __iter__() . This design ensures that functions accepting general iterables (such as for item in x: ) can accept both containers (like [1, 2, 3] ) and active iterators (like iter([1, 2, 3]) ) without distinction. it = iter([1, 2, 3])
- print(it is iter(it)) # True: iter() on an iterator returns the iterator itself
- 3. The Built-in iter() and next() Functions
Python provides two built-in functions that form the official API for interacting with the iteration protocol.
The iter() Function
The built-in iter() function can be called in two distinct forms:
Form 1: Single-Argument iter(iterable)
Invokes iterable.__iter__() to obtain an iterator. If __iter__() is not defined, Python attempts to use __getitem__() starting at index 0 . names = ["Alice", "Bob", "Charlie"]
- it = iter(names)
- print(next(it)) # Alice
- Form 2: Two-Argument iter(callable, sentinel)
When called with two arguments, the first argument must be a zero-argument callable (a function, method, or lambda), and the second argument is a sentinel value .
The resulting iterator calls callable() on each iteration and yields the return value. As soon as the callable returns a value equal to sentinel , the iterator raises StopIteration without yielding the sentinel. import random
# Function that returns a random integer between 1 and 6
def roll_die():
return random.randint(1, 6)
- # Iterate roll_die() until it rolls a 6 (the sentinel)
- # Rolls before 6 are produced; the 6 itself stops iteration
- die_iterator = iter(roll_die, 6)
for roll in die_iterator:
print(f"Rolled: {roll}")
This form is frequently used when reading fixed-size chunks from file streams or network sockets until an empty byte string b'' is encountered: # Read 64-byte blocks until EOF (empty bytes sentinel b'')
# with open('data.bin', 'rb') as f:
# for chunk in iter(lambda: f.read(64), b''):
- # process(chunk)
- The next() Function
- The built-in next() advances an iterator and retrieves its next value: next(iterator[, default])
Without default : Calls iterator.__next__() . If the iterator raises StopIteration , the exception propagates out.
With default : If the iterator raises StopIteration , next() intercepts the exception and returns the specified default value instead of raising an error. items = ["alpha", "beta"]
it = iter(items)
- print(next(it, "END")) # alpha
- print(next(it, "END")) # beta
- print(next(it, "END")) # END (StopIteration was caught internally)
- print(next(it, "END")) # END
- 4. How for Loops Work Under the Hood
In Python, a for loop is syntactic sugar for a while loop that orchestrates iter() , next() , and StopIteration exception handling.
Consider this standard loop: for item in [10, 20, 30]:
print(item)
Under the hood, the CPython interpreter executes the following exact operational sequence: # 1. Obtain an iterator from the iterable
_iterator = iter([10, 20, 30])
# 2. Repeatedly advance the iterator in a loop
while True:
try:
item = next(_iterator)
except StopIteration:
- # 3. Cleanly exit the loop when StopIteration is raised
- break
- # 4. Execute the loop body
- print(item)
This desugaring reveals why any custom object implementing __iter__() (or __getitem__() ) works seamlessly with for loops, comprehensions, in checks, unpacking, and functions like sum() , min() , and max() .
5. Designing Custom Iterable and Iterator Classes
There are two primary structural patterns for building custom iteration in Python classes:
Pattern A: Separate Iterable and Iterator Classes (Recommended for Reusability)
Separating the data container (Iterable) from the traversal state (Iterator) ensures that the collection can be iterated multiple times simultaneously without interference. class CountdownIterator:
"""Iterator tracking countdown traversal state."""
def __init__(self, start):
self.current = start
def __iter__(self):
return self
def __next__(self):
if self.current <= 0:
- raise StopIteration
- value = self.current
- self.current -= 1
- return value
class Countdown:
"""Iterable container representing a countdown range."""
def __init__(self, start):
self.start = start
def __iter__(self):
- # Return a brand new iterator on every iteration request
- return CountdownIterator(self.start)
- cd = Countdown(3)
- print(list(cd)) # [3, 2, 1]
- print(list(cd)) # [3, 2, 1] (Reusable! Can iterate again)
- Pattern B: Self-Contained Iterator Class (Single-Pass)
When an object is solely an iterator, it implements both __iter__ and __next__ within the same class definition: class FibonacciIterator:
"""Iterator producing Fibonacci numbers up to a maximum limit."""
def __init__(self, limit):
- self.limit = limit
- self.a = 0
- self.b = 1
- self.count = 0
def __iter__(self):
return self
def __next__(self):
if self.count >= self.limit:
- raise StopIteration
- value = self.a
- self.a, self.b = self.b, self.a + self.b
- self.count += 1
- return value
fib = FibonacciIterator(5)
for val in fib:
- print(val, end=" ") # 0 1 1 2 3
- print()
- 6. Iterator Exhaustion and Single-Pass Streams
A critical property tested on the PCAP exam is iterator exhaustion .
Containers (like list , tuple , set , dict ) are reusable iterables . Every time iter(container) is called, a new iterator object is instantiated at the beginning of the sequence.
Iterators (like generator objects, map , filter , zip , enumerate , or custom iterators) are stateful streams . They can only traverse forward and cannot be rewound.
Once an iterator has raised StopIteration , it is exhausted . Passing an exhausted iterator to a subsequent for loop or constructor yields zero elements: raw_data = [1, 2, 3, 4]
- # list_iterator is an iterator
- it = iter(raw_data)
- # First pass consumes all items
- first_pass = list(it)
- print("First pass:", first_pass) # [1, 2, 3, 4]
- # Second pass on the SAME iterator finds it already exhausted
- second_pass = list(it)
- print("Second pass:", second_pass) # [] (Empty!) # Built-in functional tools return single-pass iterators
- m = map(lambda x: x * 2, [1, 2, 3])
- print(1 in m) # False (m checked 2, 4, 6 and became exhausted)
- print(list(m)) # [] (Nothing left!)