25.1 Iterables

An iterable is an object that can provide its items one at a time. Lists, tuples, strings, dictionaries, sets, and range objects are common Python iterables. When you use a for loop, Python automatically requests each item from the iterable until no items remain.

An iterable is not necessarily an iterator itself. It is usually an object that can create an iterator. For example, a list stores all its values, while an iterator remembers the current position while moving through those values. Iterables make it possible to process collections in a clear and organized way.

Example

# A list is an iterable
```

colors = ["red", "green", "blue"]

# A string is also an iterable

word = "Python"

# Loop through the list

for color in colors:
print(color)

# Loop through the characters of the string

for letter in word:
print(letter)

Output

red
```

green
blue
P
y
t
h
o
n

Output Explanation

The first loop processes each value stored in the list. The second loop processes each character in the string. Python treats both objects as iterables because their contents can be accessed one item at a time.

25.2 Iterators

An iterator is an object that produces one item at a time and remembers where it is in a sequence. Unlike a list, an iterator normally does not return all its values at once. It gives the next available value only when the program asks for it.

Iterators are useful when working with large amounts of information because the program can process one value at a time. Once an iterator moves forward, it normally cannot automatically move backward or restart. A new iterator must usually be created to begin again.

Example

numbers = [10, 20, 30]
```

# Create an iterator from the list

number_iterator = iter(numbers)

# Request values one at a time

print(next(number_iterator))
print(next(number_iterator))
print(next(number_iterator))

Output

10
```

20
30

Output Explanation

Each call to next() returns the next item from the iterator. The iterator first returns 10, then 20, and finally 30. It remembers its position between each call.

25.3 The Iterator Protocol

The iterator protocol is the set of rules that an object must follow to behave as an iterator. An iterator must provide an __iter__() method and a __next__() method. These special methods allow Python to request values from the object.

The __iter__() method normally returns the iterator object itself. The __next__() method returns the next available value. When no values remain, __next__() must raise the StopIteration exception so Python knows that iteration has finished.

Example

numbers = [1, 2, 3]
```

iterator = iter(numbers)

# Python's next() calls the iterator's **next**() method

print(iterator.**next**())
print(iterator.**next**())
print(iterator.**next**())

Output

1
```

2
3

Output Explanation

Calling iterator.__next__() directly returns the same values as calling next(iterator). In normal programs, the built-in next() function is preferred because it is clearer and easier to read.

25.4 iter()

The built-in iter() function creates an iterator from an iterable. It asks the iterable for an object that can return its values one at a time. Lists, tuples, strings, sets, dictionaries, and range objects can all be passed to iter().

Creating an iterator does not normally copy the entire collection. Instead, the iterator keeps track of the current position. This allows the program to move through the original iterable in an organized way.

Example

message = "Hello"
```

# Convert the string into an iterator

character_iterator = iter(message)

print(next(character_iterator))
print(next(character_iterator))
print(next(character_iterator))

Output

H
```

e
l

Output Explanation

The iter() function creates an iterator for the string. Each call to next() returns the next character. Only the first three characters are requested, so the iterator still has two remaining characters.

25.5 next()

The built-in next() function asks an iterator for its next item. Every time it is called, the iterator advances to the following position. If the iterator still has a value, that value is returned.

The next() function can also receive a default value. If the iterator is exhausted, the default is returned instead of raising StopIteration. This can be useful when manually processing iterator values.

Example

names = iter(["Sara", "Michael"])
```

print(next(names))
print(next(names))

# Return a default value when no items remain

print(next(names, "No more names"))

Output

Sara
```

Michael
No more names

Output Explanation

The first two calls return the two available names. The third call occurs after the iterator has been exhausted. Because a default value was provided, Python returns the message instead of raising an exception.

25.6 StopIteration

StopIteration is a special exception that signals the end of an iterator. When there are no more values, the iterator raises this exception. A for loop handles it automatically and stops without displaying an error.

When you call next() manually, however, you may need to catch StopIteration yourself. Another choice is to provide a default value to next(). Understanding this exception is important when creating custom iterators.

Example

values = iter([100, 200])
```

try:
print(next(values))
print(next(values))
print(next(values))

except StopIteration:
print("The iterator has no more values.")

Output

100
```

200
The iterator has no more values.

Output Explanation

The iterator contains only two values. The third call to next() raises StopIteration. The except block catches the exception and displays a helpful message instead of allowing the program to stop unexpectedly.

25.7 Creating Custom Iterators

You can create your own iterator by defining a class with __iter__() and __next__() methods. The class must remember its current state so it knows which value to return next.

The __next__() method should update the state after returning each value. When the iterator reaches its limit, it must raise StopIteration. Custom iterators are useful when values follow a special pattern or come from a custom data source.

Example

class CountUp:
def __init__(self, start, end):
    # Save the starting and ending values
    self.current = start
    self.end = end

def __iter__(self):
    # Return this object as the iterator
    return self

def __next__(self):
    # Stop when the current value passes the ending value
    if self.current > self.end:
        raise StopIteration

    # Save the current value
    value = self.current

    # Move to the next value
    self.current += 1

    return value
```

counter = CountUp(1, 5)

for number in counter:
print(number)

Output

1
```

2
3
4
5

Output Explanation

The iterator begins at 1 and increases its current value after each step. When the current value becomes greater than 5, it raises StopIteration. The for loop catches that signal and stops automatically.

25.8 Infinite Iterators

An infinite iterator continues producing values without a natural ending. It does not raise StopIteration unless the programmer adds a special stopping condition. Infinite iterators can create number sequences, repeated patterns, or continuous data streams.

Infinite iterators must be used carefully because an unrestricted loop may continue forever. Programs normally stop them with a condition, a counter, a break statement, or a function such as itertools.islice().

Example

class EvenNumbers:
def __init__(self):
    self.current = 0

def __iter__(self):
    return self

def __next__(self):
    value = self.current
    self.current += 2
    return value
```

even_iterator = EvenNumbers()

# Request only the first five values

for count in range(5):
print(next(even_iterator))

Output

0
```

2
4
6
8

Output Explanation

The custom iterator can continue producing even numbers forever because it never raises StopIteration. The surrounding range(5) loop safely limits the example to five values.

25.9 Lazy Evaluation

Lazy evaluation means creating or calculating a value only when it is needed. Iterators use this approach because they normally produce one item at a time instead of preparing every possible result in advance.

Lazy evaluation can save time and memory, especially when a program may use only a few values from a very large sequence. It also makes it possible to work with infinite sequences because the program does not attempt to create all values at once.

Example

# map() returns an iterator in Python 3
```

numbers = [1, 2, 3, 4, 5]

squared_numbers = map(lambda number: number ** 2, numbers)

# Values are produced only when requested

print(next(squared_numbers))
print(next(squared_numbers))

# Process the remaining values

for value in squared_numbers:
print(value)

Output

1
```

4
9
16
25

Output Explanation

The map() function does not immediately create a complete list of squares. It calculates each square when the program requests it. The first two values are requested manually, and the loop processes the remaining values.

25.10 Iterator Memory Efficiency

A list stores all its elements in memory at the same time. An iterator usually stores only enough information to produce the next value. This difference can be important when working with millions of records or large files.

The range() object is memory efficient because it calculates numbers as needed instead of storing every number. Iterator-based tools are often preferred for large datasets, file processing, database results, and data pipelines.

Example

import sys
```

# A list stores one million numbers

number_list = list(range(1000000))

# A range object describes the same sequence

number_range = range(1000000)

print("List size:", sys.getsizeof(number_list))
print("Range size:", sys.getsizeof(number_range))

Example Output

List size: 8000056
```

Range size: 48

Output Explanation

The exact memory values may vary between Python versions and computers. However, the list normally uses far more memory because it stores one million references. The range object stores only the information needed to generate the sequence.

25.11 The itertools Module

The itertools module contains efficient tools for creating and combining iterators. These functions can repeat values, connect sequences, group data, create combinations, generate accumulating totals, and select specific portions of an iterator.

Most itertools functions use lazy evaluation. They return iterators instead of complete lists. This makes them useful for large collections, but you may need to convert the result to a list when you want to display every value at once.

Example

from itertools import chain
```

first_group = [1, 2, 3]
second_group = [4, 5, 6]

# Join the two iterables without creating a new combined list first

combined = chain(first_group, second_group)

for number in combined:
print(number)

Output

1
```

2
3
4
5
6

Output Explanation

The chain() function processes all values from the first iterable and then all values from the second. It presents them as one continuous iterator without first combining them into another complete list.

25.12 Infinite Iterators in itertools

The itertools module provides three common infinite iterator functions: count(), cycle(), and repeat(). The count() function creates an endless number sequence, while cycle() repeatedly loops through an iterable.

The repeat() function returns the same value repeatedly. It can be infinite, or it can receive a number that limits how many times the value is produced. Infinite iterators should always be controlled by a stopping condition.

Example 1: count()

from itertools import count
```

# Start at 10 and increase by 5

counter = count(10, 5)

for _ in range(5):
print(next(counter))

Output

10
```

15
20
25
30

Example 2: cycle()

from itertools import cycle
```

traffic_lights = cycle(["Red", "Green", "Yellow"])

for _ in range(7):
print(next(traffic_lights))

Output

Red
```

Green
Yellow
Red
Green
Yellow
Red

Example 3: repeat()

from itertools import repeat
```

messages = repeat("Welcome", 3)

for message in messages:
print(message)

Output

Welcome
```

Welcome
Welcome

Output Explanation

The first iterator produces numbers that increase by five. The second repeatedly cycles through three traffic-light values. The third repeats the word Welcome exactly three times because a repetition limit was provided.

25.13 Combinatoric Iterators

Combinatoric iterators create different arrangements or selections from a group of values. The main functions are product(), permutations(), combinations(), and combinations_with_replacement().

A permutation considers order, so A, B and B, A are different. A combination does not consider order, so those two selections are treated as the same. A Cartesian product combines every value from one group with every value from another.

Example 1: Permutations

from itertools import permutations
```

letters = ["A", "B", "C"]

results = permutations(letters, 2)

for item in results:
print(item)

Output

('A', 'B')
```

('A', 'C')
('B', 'A')
('B', 'C')
('C', 'A')
('C', 'B')

Example 2: Combinations

from itertools import combinations
```

students = ["Ali", "Sara", "Michael"]

teams = combinations(students, 2)

for team in teams:
print(team)

Output

('Ali', 'Sara')
```

('Ali', 'Michael')
('Sara', 'Michael')

Example 3: Product

from itertools import product
```

sizes = ["Small", "Large"]
colors = ["Red", "Blue"]

options = product(sizes, colors)

for option in options:
print(option)

Output

('Small', 'Red')
```

('Small', 'Blue')
('Large', 'Red')
('Large', 'Blue')

Output Explanation

Permutations include every ordered arrangement of two letters. Combinations create unique two-person teams without reversing the same pair. Product creates every possible pairing between the available sizes and colors.

25.14 Terminating Iterators

Terminating iterators eventually stop after processing a limited amount of data. The itertools module includes tools such as accumulate(), chain(), compress(), dropwhile(), takewhile(), filterfalse(), and islice().

These tools help select, combine, skip, limit, or calculate values while keeping iterator behavior. They are called terminating iterators because they finish when their input ends or when a stopping condition is reached.

Example 1: accumulate()

from itertools import accumulate
```

numbers = [10, 20, 30, 40]

running_totals = accumulate(numbers)

print(list(running_totals))

Output

[10, 30, 60, 100]

Example 2: islice()

from itertools import count, islice
```

numbers = count(1)

# Take values from positions 0 through 4

first_five = islice(numbers, 5)

print(list(first_five))

Output

[1, 2, 3, 4, 5]

Example 3: takewhile()

from itertools import takewhile
```

numbers = [2, 4, 6, 9, 10, 12]

result = takewhile(lambda number: number % 2 == 0, numbers)

print(list(result))

Output

[2, 4, 6]

Output Explanation

The first example creates running totals. The second safely takes five values from an infinite counter. The third returns values while they are even and stops as soon as it reaches the first odd number.

25.15 Practical Iterator Applications

Iterators are useful for processing files, large datasets, database results, sensor readings, paginated information, and streamed data. They let a program handle one item at a time instead of loading everything into memory.

In this example, a custom iterator processes a list of sales in fixed-size batches. Batch processing is common when sending records to a server, preparing reports, importing database information, or processing large collections in manageable groups.

Example: Batch Iterator

class BatchIterator:
def __init__(self, data, batch_size):
    self.data = data
    self.batch_size = batch_size
    self.position = 0

def __iter__(self):
    return self

def __next__(self):
    # Stop when every item has been processed
    if self.position >= len(self.data):
        raise StopIteration

    # Calculate the ending position of the next batch
    end_position = self.position + self.batch_size

    # Get the batch
    batch = self.data[self.position:end_position]

    # Move the position forward
    self.position = end_position

    return batch
```

sales = [125, 300, 175, 450, 220, 510, 90]

batches = BatchIterator(sales, 3)

for batch in batches:
print("Processing batch:", batch)
print("Batch total:", sum(batch))

Output

Processing batch: [125, 300, 175]
```

Batch total: 600
Processing batch: [450, 220, 510]
Batch total: 1180
Processing batch: [90]
Batch total: 90

Output Explanation

The iterator processes three sales at a time. The final batch contains only one value because no additional records remain. Each batch can be handled separately without requiring the program to process the entire collection at once.

25.16 Chapter Practice Exercises

These exercises reinforce the main iterator concepts from this chapter. Begin with built-in iterables and iterator functions, and then practise custom iterator classes and the tools provided by itertools.

1. Create a list of five fruits and loop through it.
2. Create an iterator from a tuple and request each value with next().
3. Create an iterator from the string Python.
4. Use a default value with next() after an iterator is exhausted.
5. Catch StopIteration using try and except.
6. Create a custom iterator that counts from 5 to 10.
7. Create a custom iterator that counts backward from 10 to 1.
8. Create an iterator that returns square numbers.
9. Create an infinite iterator that produces multiples of 3.
10. Limit an infinite iterator to ten values.
11. Use map() to lazily double a list of numbers.
12. Compare the memory usage of a list and a range object.
13. Use itertools.chain() to connect three lists.
14. Use itertools.count() to generate five values beginning at 100.
15. Use itertools.cycle() to repeat three colors.
16. Use itertools.repeat() to repeat a message four times.
17. Create every two-letter permutation of A, B, and C.
18. Create every two-person combination from four names.
19. Use product() to combine sizes and colors.
20. Use accumulate() to calculate running totals.
21. Use islice() to select the first ten values from an infinite iterator.
22. Use takewhile() to process values until one reaches 100.
23. Create a batch iterator that returns two values at a time.

Practice Example: Countdown Iterator

class Countdown:
def __init__(self, start):
    self.current = start

def __iter__(self):
    return self

def __next__(self):
    if self.current < 1:
        raise StopIteration

    value = self.current
    self.current -= 1
    return value
```

countdown = Countdown(5)

for number in countdown:
print(number)

print("Finished!")

Output

5
```

4
3
2
1
Finished!

Output Explanation

The iterator begins at five and decreases its current value after each step. When the current value becomes less than one, it raises StopIteration. The loop then finishes and displays the final message.

25.17 Chapter Mini Project

Project: Student Record Batch Processor

In this mini project, you will create a custom iterator that processes student records in batches. Each student record contains a name and a score. The iterator returns a small group of students at a time so the program can calculate statistics for each batch.

This project combines classes, the iterator protocol, lists, dictionaries, slicing, loops, calculations, and StopIteration. It demonstrates how real applications can divide large collections into smaller and more manageable pieces.

Complete Program

class StudentBatchIterator:
def __init__(self, students, batch_size):
    # Store the complete collection
    self.students = students

    # Store the requested batch size
    self.batch_size = batch_size

    # Begin at the first record
    self.position = 0

def __iter__(self):
    # This object acts as its own iterator
    return self

def __next__(self):
    # Stop when every student has been processed
    if self.position >= len(self.students):
        raise StopIteration

    # Calculate where the batch should end
    end_position = self.position + self.batch_size

    # Select the next batch
    batch = self.students[self.position:end_position]

    # Move the position forward
    self.position = end_position

    return batch
```

def calculate_batch_average(batch):
# Add all student scores
total = sum(student["score"] for student in batch)

```
# Divide by the number of students
return total / len(batch)
```

students = [
{"name": "Ali", "score": 85},
{"name": "Sara", "score": 92},
{"name": "Michael", "score": 78},
{"name": "Emma", "score": 88},
{"name": "David", "score": 95},
{"name": "Nora", "score": 81},
{"name": "Daniel", "score": 90}
]

# Process three students at a time

student_batches = StudentBatchIterator(students, 3)

batch_number = 1
overall_total = 0
student_count = 0

for batch in student_batches:
print("Batch", batch_number)
print("--------------------")

```
for student in batch:
    print(student["name"], "-", student["score"])

    overall_total += student["score"]
    student_count += 1

batch_average = calculate_batch_average(batch)

print("Batch average:", round(batch_average, 2))
print()

batch_number += 1
```

# Calculate the overall class average

overall_average = overall_total / student_count

print("Overall class average:", round(overall_average, 2))

Output

Batch 1
```

---

Ali - 85
Sara - 92
Michael - 78
Batch average: 85.0

## Batch 2

Emma - 88
David - 95
Nora - 81
Batch average: 88.0

## Batch 3

Daniel - 90
Batch average: 90.0

Overall class average: 87.0

Project Explanation

The StudentBatchIterator class receives the student collection and the number of records that should appear in each batch. The position attribute remembers where the next batch should begin.

The __iter__() method returns the iterator object. The __next__() method checks whether all students have been processed. If they have, it raises StopIteration. Otherwise, it uses list slicing to return the next group.

The calculate_batch_average() function adds the scores in one batch and divides the total by the batch size. The main loop also calculates the overall score total and student count so the complete class average can be displayed at the end.

The final batch contains only one student because the total number of students is not evenly divisible by three. The iterator handles this automatically because list slicing returns all remaining values without causing an error.

How to Run the Mini Project

1. Open Visual Studio Code, IDLE, PyCharm, or another Python editor.
2. Create a new file named student_batch_processor.py.
3. Copy the complete project code into the file.
4. Save the file.
5. Open a terminal in the folder containing the file.
6. Run python student_batch_processor.py.
7. On some computers, run python3 student_batch_processor.py.
8. Review each batch and its calculated average.
9. Review the overall class average at the end.

Project Challenges

• Ask the user to choose the batch size.
• Read student records from a text or CSV file.
• Display the highest score in each batch.
• Display the lowest score in each batch.
• Show only students who passed.
• Sort students before creating the iterator.
• Add a letter grade to each student.
• Save the batch report to a file.
• Create an iterator that processes one student at a time.
• Use itertools.islice() to create another batch-processing solution.