Generators and Iterators

A generator is a function that produces values one at a time with yield, pausing between them. Because it doesn't build the whole list in memory, it can process a million-line file or an endless stream using almost no memory. Generators are the easiest way to make anything work with foreach.

yield

A function containing yield returns a Generator object. Each time foreach asks for the next value, the function runs until the next yield, hands over that value, and pauses:

PHP 
<?php

function countdown(int $from): Generator
{
    echo "starting\n";
    for ($i = $from; $i > 0; $i--) {
        echo "yielding $i\n";
        yield $i;
    }
    echo "finished\n";
}

foreach (countdown(3) as $n) {
    echo "got $n\n";
}
Output
starting
yielding 3
got 3
yielding 2
got 2
yielding 1
got 1
finished

The function and the loop take turns. Nothing runs at all until the loop starts asking for values.

Saving memory

Compare building a big array with generating the values lazily:

PHP 
<?php

function rangeArray(int $n): array
{
    $result = [];
    for ($i = 1; $i <= $n; $i++) {
        $result[] = $i;
    }
    return $result;
}

function rangeGenerator(int $n): Generator
{
    for ($i = 1; $i <= $n; $i++) {
        yield $i;
    }
}

$start = memory_get_usage();
$sum = 0;
foreach (rangeArray(1_000_000) as $i) { $sum += $i; }
$arrayPeak = memory_get_peak_usage() - $start;

echo "Sum: $sum\n";
echo "Array version used about ", round($arrayPeak / 1024 / 1024), " MB\n";

$sum = 0;
$before = memory_get_usage();
foreach (rangeGenerator(1_000_000) as $i) { $sum += $i; }
echo "Generator version used ", memory_get_usage() - $before < 10_000 ? "almost nothing" : "a lot", "\n";
Output
Sum: 500000500000
Array version used about 16 MB
Generator version used almost nothing

Reading large files

The classic use: processing a file line by line without loading it all into memory:

PHP 
<?php

function readLines(string $path): Generator
{
    $handle = fopen($path, 'r');
    try {
        while (($line = fgets($handle)) !== false) {
            yield rtrim($line, "\n");
        }
    } finally {
        fclose($handle);
    }
}

file_put_contents("access.log", implode("\n", [
    "GET /home 200", "GET /missing 404", "POST /login 200", "GET /old 404", "GET /shop 500",
]) . "\n");

$errors = 0;
foreach (readLines("access.log") as $line) {
    if (preg_match('/ (4|5)\d\d$/', $line)) {
        $errors++;
        echo "error: $line\n";
    }
}
echo "$errors errors\n";
Output
error: GET /missing 404
error: GET /old 404
error: GET /shop 500
3 errors

The finally block guarantees the file is closed, even if the loop stops early.

Keys, infinite sequences and composing

yield $key => $value produces keys too. Generators can be infinite — the consumer decides when to stop — and one generator can wrap another to build processing pipelines:

PHP 
<?php

function fibonacci(): Generator
{
    [$a, $b] = [0, 1];
    while (true) {
        yield $a;
        [$a, $b] = [$b, $a + $b];
    }
}

function take(iterable $items, int $n): Generator
{
    if ($n <= 0) {
        return;
    }
    foreach ($items as $key => $item) {
        yield $key => $item;
        if (--$n === 0) {
            return;
        }
    }
}

function filterIt(iterable $items, callable $keep): Generator
{
    foreach ($items as $key => $item) {
        if ($keep($item)) {
            yield $key => $item;
        }
    }
}

echo implode(", ", iterator_to_array(take(fibonacci(), 10), false)), "\n";

$evenFibs = take(filterIt(fibonacci(), fn($n) => $n % 2 === 0), 5);
foreach ($evenFibs as $position => $value) {
    echo "F($position) = $value\n";
}
Output
0, 1, 1, 2, 3, 5, 8, 13, 21, 34
F(0) = 0
F(3) = 2
F(6) = 8
F(9) = 34
F(12) = 144

iterator_to_array collects a generator's values into an array; pass false to ignore the keys.

yield from

yield from passes along every value of another generator or array — handy for recursion over trees:

PHP 
<?php

function walk(array $tree, string $prefix = ""): Generator
{
    foreach ($tree as $name => $children) {
        yield $prefix . $name;
        yield from walk($children, "$prefix$name/");
    }
}

$folders = ["src" => ["Models" => [], "Http" => ["Controllers" => []]], "tests" => []];
foreach (walk($folders) as $path) {
    echo $path, "\n";
}
Output
src
src/Models
src/Http
src/Http/Controllers
tests

Making a class iterable

To let foreach loop over your own objects, implement IteratorAggregate. With a generator, getIterator is usually three lines:

PHP 
<?php

final class Playlist implements IteratorAggregate
{
    private array $tracks = [];

    public function add(string $title, int $seconds): void
    {
        $this->tracks[] = [$title, $seconds];
    }

    public function getIterator(): Generator
    {
        foreach ($this->tracks as [$title, $seconds]) {
            yield $title => sprintf("%d:%02d", intdiv($seconds, 60), $seconds % 60);
        }
    }
}

$list = new Playlist();
$list->add("Drive", 245);
$list->add("Holiday", 213);

foreach ($list as $title => $length) {
    echo str_pad($title, 8), $length, "\n";
}
Output
Drive   4:05
Holiday 3:33

Iterators and iterables

Under the hood, foreach works with anything Traversable: arrays, generators, and objects implementing Iterator (five methods: current, key, next, rewind, valid) or IteratorAggregate. The iterable type accepts arrays and traversables alike. The SPL library includes ready-made iterators such as ArrayIterator, LimitIterator and RecursiveDirectoryIterator.

One limitation: a generator can only be iterated once. To loop again, call the generator function again.

Try it yourself

Write a generator that yields batches of a given size from any iterable — handy for inserting rows into a database in chunks.

Try it yourself · main.php
Edit the code, copy it into main.php, then run it with php main.php (PHP 8.5). How to install PHP
Output
Batch 0: 1, 2, 3
Batch 1: 4, 5, 6
Batch 2: 7
PreviousErrors and Exceptions