Parallelism and Thread Safety

Modern computers have many processor cores. Parallelism splits CPU-heavy work across them so it finishes sooner. But as soon as several threads touch the same data at once, a new class of bugs appears: race conditions. This lesson shows how to run work in parallel with Parallel, PLINQ and Task.Run, and how to keep shared data safe.

Concurrency, parallelism and async

• Async (last lesson) is about waiting efficiently for I/O without holding threads.
• Parallelism is about computing on several cores at the same time.

Downloading 100 web pages is an async problem; resizing 100 images is a parallel one.

Parallel.For and Parallel.ForEach

These look like loops, but iterations are spread across threads and run in no particular order:

var results = new int[10];

Parallel.For(0, results.Length, i =>
{
    results[i] = SlowSquare(i);
});

Console.WriteLine(string.Join(", ", results));

static int SlowSquare(int n)
{
    Thread.Sleep(50);    // pretend this is heavy work
    return n * n;
}

0, 1, 4, 9, 16, 25, 36, 49, 64, 81

This is safe because each iteration writes to its own array slot. If you printed from inside the loop, the lines would come out in a different order each run.

Parallel.ForEachAsync combines both worlds: it runs async work for each item with a limited number in flight at once — for example downloading many files, but no more than four at a time.

Race conditions

Here is the classic bug. Many threads add to the same counter:

int counter = 0;
Parallel.For(0, 1_000_000, _ => counter++);
Console.WriteLine(counter);     // usually much less than 1,000,000 — and different every run

counter++ is really three steps: read the value, add one, write it back. Two threads can read the same value, both add one, and both write the same result — one increment is lost. Because the outcome depends on timing, bugs like this may pass every test and then fail in production.

Fix 1: Interlocked

For simple counters, Interlocked performs the read-modify-write as a single, indivisible (atomic) operation:

int counter = 0;
Parallel.For(0, 1_000_000, _ => Interlocked.Increment(ref counter));
Console.WriteLine(counter);

1000000

Fix 2: lock

For anything more complex, a lock ensures only one thread at a time runs a block of code. Since C# 13 there's a dedicated Lock type for this:

var totals = new Dictionary<string, int>();
var gate = new Lock();
string[] words = "a b a c b a d c a b".Split(' ');

Parallel.ForEach(Enumerable.Range(0, 1000), i =>
{
    string word = words[i % words.Length];
    lock (gate)
    {
        totals[word] = totals.GetValueOrDefault(word) + 1;
    }
});

foreach (var (word, count) in totals.OrderBy(p => p.Key))
    Console.WriteLine($"{word}: {count}");

a: 400
b: 300
c: 200
d: 100

Keep locked sections short, and never await inside a lock (the compiler won't allow it). Older code locks on a private object field instead of a Lock; it works the same way.

Fix 3: thread-safe collections

System.Collections.Concurrent offers collections designed for use from many threads, such as ConcurrentDictionary, ConcurrentQueue and ConcurrentBag:

using System.Collections.Concurrent;

var counts = new ConcurrentDictionary<int, int>();

Parallel.For(1, 10_001, n =>
{
    int digits = n.ToString().Length;
    counts.AddOrUpdate(digits, 1, (_, old) => old + 1);
});

foreach (var (digits, count) in counts.OrderBy(p => p.Key))
    Console.WriteLine($"{digits}-digit numbers: {count}");

1-digit numbers: 9
2-digit numbers: 90
3-digit numbers: 900
4-digit numbers: 9000
5-digit numbers: 1

Fix 4: don't share at all

The best way to avoid races is to give each thread its own data and combine results at the end. PLINQ does exactly that: add AsParallel() to a LINQ query and it splits the work across cores, then merges the results safely:

long primeCount = Enumerable.Range(2, 2_000_000)
    .AsParallel()
    .Count(IsPrime);
Console.WriteLine(primeCount);

int[] firstPrimes = Enumerable.Range(2, 100)
    .AsParallel()
    .AsOrdered()          // keep the original order in the results
    .Where(IsPrime)
    .Take(10)
    .ToArray();
Console.WriteLine(string.Join(" ", firstPrimes));

static bool IsPrime(int n)
{
    if (n < 2) return false;
    for (int d = 2; (long)d * d <= n; d++)
        if (n % d == 0) return false;
    return true;
}

148933
2 3 5 7 11 13 17 19 23 29

Without AsOrdered, a parallel query may return results in any order.

Task.Run: moving work off the current thread

Task.Run runs CPU-bound work on a thread-pool thread and gives you a task to await. In a desktop app it keeps the UI responsive during a heavy calculation:

Task<long> sumTask = Task.Run(() =>
{
    long sum = 0;
    for (int i = 1; i <= 100_000_000; i++) sum += i % 7;
    return sum;
});

Console.WriteLine("Calculating in the background...");
Console.WriteLine($"Result: {await sumTask}");

Calculating in the background...
Result: 299999997

Don't wrap I/O in Task.Run — use real async methods for that — and in ASP.NET Core, don't use it to "make" request handlers asynchronous; it only moves the work to another thread from the same pool.

Immutability helps

Data that never changes can be read by any number of threads with no locking at all. That's one more reason to favour records, readonly fields and immutable collections (System.Collections.Immutable) for shared data.

Thread-safety checklist

• Is any data written by one thread while another reads or writes it? If not, you're safe.
• If yes: can you avoid sharing (give each thread its own data, then combine)?
• If not: use Interlocked for simple numbers, a concurrent collection for collections, or a lock for everything else.
• Standard collections like List<T> and Dictionary are not safe for concurrent writes.
• Measure before and after: for small amounts of work, the overhead of parallelism can make code slower.

Try it yourself

Estimate π by throwing random darts at a square: the fraction that land inside the quarter circle approaches π/4. Each parallel worker uses its own Random and its own count, and the counts are combined with Interlocked.Add. (Results vary slightly from run to run.)