Iterators and yield

Anything you can loop over with foreach implements IEnumerable<T>. With the yield return statement you can write your own sequences in a few lines — sequences that produce values lazily, one at a time, only when asked. That's how LINQ itself is built, and how you can process huge or even infinite streams of data.

How foreach works

foreach asks a collection for an enumerator and then repeatedly calls MoveNext() and reads Current. These two loops do the same thing:

string[] colours = ["red", "green", "blue"];

foreach (string c in colours)
    Console.Write(c + " ");
Console.WriteLine();

using IEnumerator<string> e = ((IEnumerable<string>)colours).GetEnumerator();
while (e.MoveNext())
    Console.Write(e.Current + " ");
Console.WriteLine();

Output

red green blue
red green blue

You'll almost never write the second form, but knowing it explains what yield does for you.

yield return

A method that returns IEnumerable<T> and contains yield return is an iterator. Each yield return hands one value to the caller and pauses the method; it resumes from that point when the next value is requested:

foreach (int n in Countdown(3))
    Console.WriteLine($"got {n}");

static IEnumerable<int> Countdown(int from)
{
    Console.WriteLine("starting");
    for (int i = from; i > 0; i--)
    {
        Console.WriteLine($"yielding {i}");
        yield return i;
    }
    Console.WriteLine("finished");
}

Output

starting
yielding 3
got 3
yielding 2
got 2
yielding 1
got 1
finished

The output interleaves: the method and the loop take turns. The compiler turns the method into a hidden class that remembers where it paused.

Lazy and infinite sequences

Because values are produced only on demand, an iterator can describe a sequence that never ends. The caller decides how much to take:

Console.WriteLine(string.Join(", ", Fibonacci().Take(12)));
Console.WriteLine(Fibonacci().First(n => n > 1000));
Console.WriteLine(string.Join(", ", Primes().Skip(20).Take(5)));

static IEnumerable<long> Fibonacci()
{
    long a = 0, b = 1;
    while (true)
    {
        yield return a;
        (a, b) = (b, a + b);
    }
}

static IEnumerable<int> Primes()
{
    for (int n = 2; ; n++)
    {
        bool prime = true;
        for (int d = 2; d * d <= n; d++)
            if (n % d == 0) { prime = false; break; }
        if (prime) yield return n;
    }
}

Output

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89
1597
73, 79, 83, 89, 97

Calling ToList() or Count() on an infinite sequence would never finish — always limit it with something like Take or First.

yield break

yield break ends the sequence early:

string[] lines = ["header", "data 1", "data 2", "END", "ignored", "also ignored"];
foreach (string line in UntilEnd(lines))
    Console.WriteLine(line);

static IEnumerable<string> UntilEnd(IEnumerable<string> source)
{
    foreach (string line in source)
    {
        if (line == "END") yield break;
        yield return line;
    }
}

Output

header
data 1
data 2

Writing your own LINQ-style operators

Combine iterators with extension methods and you can add operators that chain just like the built-in ones:

int[] readings = [3, 3, 3, 7, 7, 2, 2, 2, 2, 9];

Console.WriteLine(string.Join(" ", readings.WithoutRepeats()));
Console.WriteLine(string.Join(" ", readings.WithoutRepeats().EveryOther()));
Console.WriteLine(string.Join(" ", "abcdef".Pairwise().Select(p => $"{p.Item1}{p.Item2}")));

static class MyLinq
{
    public static IEnumerable<T> WithoutRepeats<T>(this IEnumerable<T> source)
    {
        bool first = true;
        T previous = default!;
        foreach (T item in source)
        {
            if (first || !EqualityComparer<T>.Default.Equals(item, previous))
                yield return item;
            previous = item;
            first = false;
        }
    }

    public static IEnumerable<T> EveryOther<T>(this IEnumerable<T> source)
    {
        bool take = true;
        foreach (T item in source)
        {
            if (take) yield return item;
            take = !take;
        }
    }

    public static IEnumerable<(T, T)> Pairwise<T>(this IEnumerable<T> source)
    {
        using var e = source.GetEnumerator();
        if (!e.MoveNext()) yield break;
        T prev = e.Current;
        while (e.MoveNext())
        {
            yield return (prev, e.Current);
            prev = e.Current;
        }
    }
}

Output

3 7 2 9
3 2
ab bc cd de ef

Making a class enumerable

To let foreach loop over your own class, implement IEnumerable<T>. With yield, GetEnumerator is often just a few lines:

var playlist = new Playlist("Road trip");
playlist.Add("Drive", 245);
playlist.Add("Holiday", 213);
playlist.Add("Roadrunner", 248);

foreach (var (title, seconds) in playlist)
    Console.WriteLine($"{title,-11}{seconds / 60}:{seconds % 60:D2}");

Console.WriteLine($"Total: {playlist.Sum(t => t.Seconds) / 60} minutes");

class Playlist(string name) : IEnumerable<(string Title, int Seconds)>
{
    private readonly List<(string, int)> _tracks = [];
    public string Name { get; } = name;

    public void Add(string title, int seconds) => _tracks.Add((title, seconds));

    public IEnumerator<(string Title, int Seconds)> GetEnumerator()
    {
        foreach (var track in _tracks)
            yield return track;
    }

    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() => GetEnumerator();
}

Output

Drive      4:05
Holiday    3:33
Roadrunner 4:08
Total: 11 minutes

Once a class implements IEnumerable<T>, every LINQ method works on it too — note the Sum. The second, non-generic GetEnumerator is required for compatibility with old pre-generics code.

Laziness has consequences

An iterator's body doesn't run until enumeration starts. That includes argument checks, which can surprise you — the exception appears later than expected:

var numbers = Repeat("x", -1);      // no exception yet!
Console.WriteLine("Created the sequence");
foreach (string s in numbers)       // now it throws
    Console.WriteLine(s);

static IEnumerable<string> Repeat(string value, int count)
{
    ArgumentOutOfRangeException.ThrowIfNegative(count);
    for (int i = 0; i < count; i++)
        yield return value;
}

Output

Created the sequence
Unhandled exception. System.ArgumentOutOfRangeException: count ('-1') must be a non-negative value. (Parameter 'count')
Actual value was -1.

To validate eagerly, put the checks in a normal method that then calls a private iterator (a local function works well). Also remember that enumerating an iterator twice runs it twice, like any deferred LINQ query.

Async streams

When producing each value involves waiting — reading lines from a network, paging through an API — an iterator can be async and return IAsyncEnumerable<T>, consumed with await foreach. Lesson 57 covers async and await.

Try it yourself

Write an iterator Range(start, end, step) for doubles and use it to print a small conversion table.

Output

  -10 °C =    14 °F
    0 °C =    32 °F
   10 °C =    50 °F
   20 °C =    68 °F
   30 °C =    86 °F
   40 °C =   104 °F

PreviousGrouping & Joins NextExceptions