Readers and Writers

io.Reader and io.Writer are tiny interfaces, but they connect almost everything in Go that handles data: files, network connections, HTTP bodies, compression, encryption, hashing, buffers and strings. Code written against them works with any source or destination, and streams data in chunks instead of loading it all into memory. This lesson covers the Read contract, the helpers in io, wrapping readers and writers, and writing your own.

The Read contract

Read(p []byte) (n int, err error) fills up to len(p) bytes of p and returns how many it read. At the end of the data it returns io.EOF. A careful caller processes the n bytes before looking at the error, because a reader may return data and io.EOF together:

GO 
package main

import (
	"errors"
	"fmt"
	"io"
	"strings"
)

func main() {
	r := strings.NewReader("streaming data")
	buf := make([]byte, 5)
	for {
		n, err := r.Read(buf)
		if n > 0 {
			fmt.Printf("%q\n", buf[:n])
		}
		if errors.Is(err, io.EOF) {
			break
		}
		if err != nil {
			fmt.Println("error:", err)
			break
		}
	}
}
Output
"strea"
"ming "
"data"

You rarely write this loop yourself — the helpers below do it — but knowing the contract explains why they work the way they do.

Helpers in io

GO 
package main

import (
	"bytes"
	"fmt"
	"io"
	"os"
	"strings"
)

func main() {
	all, _ := io.ReadAll(strings.NewReader("read everything at once"))
	fmt.Println(len(all), string(all))

	n, _ := io.Copy(os.Stdout, strings.NewReader("copied to stdout\n"))
	fmt.Println(n, "bytes copied")

	limited := io.LimitReader(strings.NewReader("only the first ten bytes"), 10)
	head, _ := io.ReadAll(limited)
	fmt.Printf("%q\n", head)

	var a, b bytes.Buffer
	w := io.MultiWriter(&a, &b) // every write goes to both
	io.WriteString(w, "twice")
	fmt.Println(a.String(), b.String())

	r := io.MultiReader(strings.NewReader("one "), strings.NewReader("two"))
	joined, _ := io.ReadAll(r)
	fmt.Println(string(joined))
}
Output
23 read everything at once
copied to stdout
17 bytes copied
"only the f"
twice twice
one two
HelperDoes
io.ReadAll(r)Reads until EOF into a []byte
io.Copy(w, r)Streams everything from r to w in chunks
io.LimitReader(r, n)A reader that stops after n bytes — protects against huge inputs
io.TeeReader(r, w)Copies everything read from r into w as well
io.MultiWriter, io.MultiReaderWrite to several writers; read several readers one after another
io.WriteString(w, s)Writes a string

Streaming through layers

Readers and writers stack. This program compresses text with gzip and computes a checksum while writing — one pass, no matter how big the data:

GO 
package main

import (
	"bytes"
	"compress/gzip"
	"crypto/sha256"
	"fmt"
	"io"
	"strings"
)

func main() {
	original := strings.Repeat("Go streams data through layers. ", 200)

	var compressed bytes.Buffer
	hash := sha256.New()
	zw := gzip.NewWriter(io.MultiWriter(&compressed, hash))
	io.Copy(zw, strings.NewReader(original))
	zw.Close() // flushes the compressor: required

	fmt.Println(len(original), "->", compressed.Len(), "bytes")
	fmt.Printf("checksum of compressed data starts %x\n", hash.Sum(nil)[:4])

	zr, err := gzip.NewReader(&compressed)
	if err != nil {
		fmt.Println(err)
		return
	}
	back, _ := io.ReadAll(zr)
	fmt.Println("round trip ok:", string(back) == original)
}
Output
6400 -> 84 bytes
checksum of compressed data starts 5ec73374
round trip ok: true

Replace the buffer with a file, or with an HTTP response, and nothing else changes. That's the power of programming against interfaces.

bufio: buffering and convenience

Each Read or Write on a file or network connection is a system call, so many small ones are slow. bufio.Reader and bufio.Writer wrap another reader or writer with a buffer, and add handy methods such as ReadString, ReadRune and Peek:

GO 
package main

import (
	"bufio"
	"fmt"
	"os"
	"strings"
)

func main() {
	r := bufio.NewReader(strings.NewReader("key=value\nnext line"))
	peek, _ := r.Peek(3)
	fmt.Printf("peeked %q\n", peek)
	line, _ := r.ReadString('\n')
	fmt.Printf("line %q\n", line)

	w := bufio.NewWriter(os.Stdout)
	defer w.Flush() // without Flush, buffered output can be lost
	for i := range 3 {
		fmt.Fprintf(w, "buffered line %d\n", i)
	}
}
Output
peeked "key"
line "key=value\n"
buffered line 0
buffered line 1
buffered line 2

Writing your own Reader

A custom reader can transform data on the fly. This one wraps another reader and uppercases the letters passing through:

GO 
package main

import (
	"io"
	"os"
	"strings"
	"unicode"
)

type upperReader struct{ src io.Reader }

func (u upperReader) Read(p []byte) (int, error) {
	n, err := u.src.Read(p)
	for i := range n {
		p[i] = byte(unicode.ToUpper(rune(p[i])))
	}
	return n, err
}

func main() {
	r := upperReader{strings.NewReader("quiet text becomes loud\n")}
	io.Copy(os.Stdout, r)
}
Output
QUIET TEXT BECOMES LOUD

(This simple version only handles ASCII letters correctly, since a multi-byte character could be split across two Read calls.)

Connecting a writer to a reader: io.Pipe

io.Pipe returns a connected reader and writer: whatever one goroutine writes, another reads. It's useful when one API wants to write (like an encoder) and another wants to read (like an HTTP request body):

GO 
package main

import (
	"encoding/json"
	"fmt"
	"io"
)

func main() {
	pr, pw := io.Pipe()
	go func() {
		err := json.NewEncoder(pw).Encode(map[string]int{"items": 3})
		pw.CloseWithError(err) // nil error: a normal EOF for the reader
	}()
	data, _ := io.ReadAll(pr)
	fmt.Print(string(data))
}
Output
{"items":3}

Try it yourself

Count the bytes and lines flowing through io.Copy with a counting writer, while still printing the text.

Try it yourself · main.go
Edit the code, copy it into main.go in a new folder and run it with go run main.go (Go 1.27 or later). How to install Go
Output
alpha
beta
gamma
17 bytes, 3 lines
Next: encoding and decoding JSON.
PreviousFiles and Directories