Go bufio
last modified July 20, 2026
In this tutorial we explore buffered input and output operations in Go using
the built-in bufio package.
The bufio package
Every read and write operation in Go involves a system call into the kernel.
System calls are expensive compared to in-memory operations — each one
requires a context switch between user space and kernel space. The
bufio package reduces this overhead by wrapping an
io.Reader or io.Writer with an in-memory buffer.
Think of it like carrying water from a well. Without a bucket, you would walk back and forth carrying a single cup each trip (many trips, much effort). With a bucket, you fill it once and carry many cups' worth in a single trip (fewer trips, less effort). The buffer is the bucket; the system calls are the trips.
The package provides three main types:
bufio.Reader |
Buffered reader — wraps an io.Reader and reads
from it in large chunks, then serves smaller portions from its
internal buffer on demand. |
bufio.Writer |
Buffered writer — wraps an io.Writer and
accumulates data in its internal buffer, writing it to the underlying
writer in one batch. |
bufio.Scanner |
Convenient token reader — built on top of a buffered reader, it reads input as tokens (lines, words, runes, or custom delimiters) without manual buffer management. |
Buffered reader
A buffered reader is created with bufio.NewReader or
bufio.NewReaderSize.
func NewReader(rd io.Reader) *Reader // buffer size: 4096 bytes func NewReaderSize(rd io.Reader, size int) *Reader
NewReader returns a reader whose internal buffer has the default
size of 4096 bytes (4 KB). NewReaderSize allows specifying a
custom buffer size; the actual buffer will be at least as large as the given
size.
Once created, the buffered reader reads large blocks from the underlying
io.Reader into its buffer and then serves individual reads (such
as ReadString, ReadBytes, or Peek)
from that buffer until it is exhausted, at which point it refills automatically.
Buffered writer
A buffered writer is created with bufio.NewWriter or
bufio.NewWriterSize.
func NewWriter(w io.Writer) *Writer // buffer size: 4096 bytes func NewWriterSize(w io.Writer, size int) *Writer
NewWriter returns a writer with a 4 KB internal buffer.
NewWriterSize sets a custom buffer size.
Data written to a buffered writer accumulates in the buffer. When the buffer
is full, the writer automatically flushes its contents to the underlying
io.Writer. If the buffer is not yet full, the data must be
explicitly flushed with Flush.
func (b *Writer) Flush() error
Flush writes any buffered data to the underlying io.Writer.
Forgetting to call Flush is a common pitfall — data that
never fills the buffer is silently lost when the writer is garbage collected.
Scanner
A scanner is created with bufio.NewScanner.
func NewScanner(r io.Reader) *Scanner
NewScanner returns a Scanner that reads from the given
io.Reader. The scanner internally uses a buffered reader and
splits the input into tokens using a split function, which defaults to
ScanLines (splitting on newline characters).
The Scanner is best suited for structured text data — files
with lines, comma-separated values, or custom-delimited formats. For raw
binary reading, bufio.Reader is more appropriate.
Go Reader.ReadString
The ReadString reads until the first occurrence of the given
delimiter in the input.
func (b *Reader) ReadString(delim byte) (string, error)
It returns a string containing the data up to and including the delimiter.
package main
import (
"bufio"
"fmt"
"log"
"os"
"strings"
)
func main() {
fmt.Print("Enter your name: ")
r := bufio.NewReader(os.Stdin)
name, err := r.ReadString('\n')
if err != nil {
log.Fatal(err)
}
fmt.Printf("Hello %s!\n", strings.TrimSpace(name))
}
With ReadString function, we read an input from the user and
produce a message to the console.
r := bufio.NewReader(os.Stdin)
We create a new reader from the standard input.
name, err := r.ReadString('\n')
A string input is read from the user.
$ go run main.go Enter your name: Jan Hello Jan!
Go Writer.WriteString
The WriteString writes a string to the buffer.
func (b *Writer) WriteString(s string) (int, error)
It returns the number of bytes written.
Tracking buffer state
The Available method returns the number of bytes left in the
buffer, and Buffered returns the number of bytes already written
to the buffer.
func (b *Writer) Available() int func (b *Writer) Buffered() int
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main() {
data := []string{"an old falcon", "misty mountains",
"a wise man", "a rainy morning"}
f, err := os.Create("words.txt")
if err != nil {
log.Fatal(err)
}
defer f.Close()
wr := bufio.NewWriter(f)
fmt.Printf("buffer size: %d\n", wr.Size())
for _, line := range data {
wr.WriteString(line)
wr.WriteString("\n")
fmt.Printf("buffered: %d\tavailable: %d\n",
wr.Buffered(), wr.Available())
}
wr.Flush()
fmt.Println("data written")
}
We write several strings to a buffered writer while monitoring the buffer state after each write.
wr := bufio.NewWriter(f)
A new writer is created with the default buffer size of 4KB. The
Size method returns the buffer size.
wr.WriteString(line)
wr.WriteString("\n")
fmt.Printf("buffered: %d\tavailable: %d\n",
wr.Buffered(), wr.Available())
After each write, we call Buffered to see how much data is sitting
in the buffer and Available to see how much space remains.
Writing the string and newline separately avoids an unnecessary string
allocation compared to line + "\n".
wr.Flush()
Since our total data (about 80 bytes) is much smaller than the default 4KB
buffer, we must call Flush to transfer the buffered data to the
underlying file.
$ go run main.go buffer size: 4096 buffered: 15 available: 4081 buffered: 31 available: 4065 buffered: 43 available: 4053 buffered: 59 available: 4037 data written
Forgetting to flush
If we omit the Flush call, the data remains in the buffer and is
lost when the program exits.
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main() {
f, err := os.Create("words.txt")
if err != nil {
log.Fatal(err)
}
defer f.Close()
wr := bufio.NewWriter(f)
wr.WriteString("an old falcon\n")
wr.WriteString("misty mountains\n")
// Flush is not called — data stays in the buffer
fmt.Println("done")
}
The example writes two strings but never calls Flush.
wr.WriteString("an old falcon\n")
wr.WriteString("misty mountains\n")
// Flush is not called — data stays in the buffer
The data is written to the in-memory buffer, but since it never reaches the
4KB threshold and Flush is not called, the buffer is discarded
when the program exits. The resulting words.txt file is empty.
$ go run main.go done $ cat words.txt $
Reading file line by line with Scanner
In the next example, we read a file line by line with a Scanner.
sky nice cup cloud forest water pond lake snow
This is the words.txt file.
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main() {
f, err := os.Open("words.txt")
if err != nil {
log.Fatal(err)
}
defer f.Close()
scanner := bufio.NewScanner(f)
for scanner.Scan() {
fmt.Println(scanner.Text())
}
if err := scanner.Err(); err != nil {
log.Fatal(err)
}
}
The example reads a small file containing words on each line.
scanner := bufio.NewScanner(f)
A new scanner is created with bufio.NewScanner.
for scanner.Scan() {
fmt.Println(scanner.Text())
}
The Scan function advances the Scanner to the next token, which
will then be available through the Bytes or Text
method. By default, the function advances by lines.
Use scanner.Bytes() instead of scanner.Text() when
you want to avoid allocating a new string for each token — it returns a slice
of bytes that is only valid until the next Scan call, but avoids
the memory allocation that Text performs.
$ go run main.go sky nice cup cloud forest water pond lake snow
Read words from a string
In the following example, we read words from a string using Scanner.
package main
import (
"bufio"
"fmt"
"log"
"strings"
)
func main() {
words := []string{}
data := "A foggy mountain.\nAn old falcon.\nA wise man."
sc := bufio.NewScanner(strings.NewReader(data))
sc.Split(bufio.ScanWords)
n := 0
for sc.Scan() {
words = append(words, sc.Text())
n++
}
if err := sc.Err(); err != nil {
log.Fatal(err)
}
fmt.Printf("# of words: %d\n", n)
for _, word := range words {
fmt.Println(word)
}
}
The strings.NewReader returns a new reader from a string.
sc.Split(bufio.ScanWords)
We tell the scanner to scan by words using Split.
The ScanWords split function splits the input on whitespace
boundaries (spaces, tabs, newlines), which makes it useful for parsing
space-delimited text.
Go Scanner custom split function
We can create a custom split function with SplitFunc.
type SplitFunc func(data []byte, atEOF bool) (advance int, token []byte, err error)
The function receives the data slice and a boolean indicating whether the end of the input has been reached. It returns the number of bytes to advance, the token to be returned to the client, and any error encountered.
package main
import (
"bufio"
"bytes"
"fmt"
"log"
"strings"
)
func main() {
data := "sky, cup, cloud, forest, water, pond"
sc := bufio.NewScanner(strings.NewReader(data))
sc.Split(scanComma)
for sc.Scan() {
fmt.Println(sc.Text())
}
if err := sc.Err(); err != nil {
log.Fatal(err)
}
}
func scanComma(data []byte, atEOF bool) (advance int, token []byte, err error) {
if atEOF && len(data) == 0 {
return 0, nil, nil
}
if i := strings.IndexRune(string(data), ','); i >= 0 {
j := i + 1
// skip whitespace after comma
for j < len(data) && data[j] == ' ' {
j++
}
return j, bytes.TrimSpace(data[:i]), nil
}
if atEOF {
return len(data), bytes.TrimSpace(data), nil
}
return 0, nil, nil
}
In the example, we create a custom scan function that splits a string by commas while trimming whitespace.
sc.Split(scanComma)
We set the custom split function with Split.
func scanComma(data []byte, atEOF bool) (advance int, token []byte, err error) {
The custom function scans for commas and returns tokens without the delimiter.
The TrimSpace call removes leading whitespace after commas.
$ go run main.go sky cup cloud forest water pond
Go Writer.WriteRune
The WriteRune writes a single rune (Unicode code point).
func (b *Writer) WriteRune(r rune) (size int, err error)
It returns the number of bytes written and any error.
In the following example, we write runes to a file and then read them back
with ScanRunes for a complete round-trip.
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main() {
runes := "🐜🐬🐄🐘🦂🐫🐑🦍🐯🐞"
f, err := os.Create("runes.txt")
if err != nil {
log.Fatal(err)
}
defer f.Close()
wr := bufio.NewWriter(f)
for _, r := range runes {
wr.WriteRune(r)
wr.WriteRune('\n')
}
wr.Flush()
fmt.Println("runes written")
}
In the first part, we write each rune on a separate line to a file.
for _, r := range runes {
wr.WriteRune(r)
wr.WriteRune('\n')
}
We iterate over the string of runes and write each one followed by a newline.
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main() {
f, err := os.Open("runes.txt")
if err != nil {
log.Fatal(err)
}
defer f.Close()
sc := bufio.NewScanner(f)
sc.Split(bufio.ScanRunes)
n := 0
for sc.Scan() {
fmt.Println(sc.Text())
n++
}
if err := sc.Err(); err != nil {
log.Fatal(err)
}
fmt.Printf("read %d runes\n", n)
}
In the second part, we open the file and read the runes back using
ScanRunes.
sc := bufio.NewScanner(f) sc.Split(bufio.ScanRunes)
We create a scanner from the file and set the split function to
ScanRunes to scan the input rune by rune.
$ go run read.go 🐜 🐬 🐄 🐘 🦂 🐫 🐑 🦍 🐯 🐞 read 10 runes
Go Reader.Peek
The Peek method returns the next n bytes without
advancing the reader.
func (b *Reader) Peek(n int) ([]byte, error)
This is useful when we need to inspect upcoming data before deciding how to read it, such as checking file headers or determining content type.
package main
import (
"bufio"
"fmt"
"log"
"strings"
)
func main() {
data := "an old falcon"
r := bufio.NewReader(strings.NewReader(data))
peeked, err := r.Peek(5)
if err != nil {
log.Fatal(err)
}
fmt.Printf("Peeked: %s\n", peeked)
word, _ := r.ReadString(' ')
fmt.Printf("Read: %s", word)
}
We peek at the first five bytes without consuming them, then read normally.
peeked, err := r.Peek(5)
The Peek method returns the first five bytes. The reader position
is not advanced yet.
word, _ := r.ReadString(' ')
We call ReadString afterward and it reads from the same starting
position, confirming that Peek did not consume the data.
$ go run main.go Peeked: an ol Read: an old
Go Reader.Read
The Reader.Read function reads data into a slice of bytes.
func (b *Reader) Read(p []byte) (n int, err error)
It returns the number of bytes read.
In the next example we also use the hex package, which implements
hexadecimal encoding and decoding.
package main
import (
"bufio"
"encoding/hex"
"fmt"
"log"
"os"
"io"
)
func main() {
f, err := os.Open("sid.jpg")
if err != nil {
log.Fatal(err)
}
defer f.Close()
reader := bufio.NewReader(f)
buf := make([]byte, 256)
for {
_, err := reader.Read(buf)
if err != nil {
if err != io.EOF {
fmt.Println(err)
}
break
}
fmt.Printf("%s", hex.Dump(buf))
}
}
In the code example, we read an image and print it in hexadecimal format.
reader := bufio.NewReader(f)
We create a reader with bufio.NewReader.
buf := make([]byte, 256)
We create a buffer of 256 bytes. The buffer size determines how many bytes are read per system call — a larger buffer means fewer calls but uses more memory. For example, a 64 KB buffer would reduce syscalls by a factor of 256 compared to 256 bytes.
for {
_, err := reader.Read(buf)
...
We read the binary data in a for loop. Each iteration fills the buffer with up
to len(buf) bytes. Using a smaller buffer increases the number of
iterations and system calls; using a larger buffer (e.g. 4096 or 65536 bytes)
reduces overhead for large files.
fmt.Printf("%s", hex.Dump(buf))
The Dump returns a string that contains a hex dump of the given
data.
$ go run main.go 00000000 ff d8 ff e0 00 10 4a 46 49 46 00 01 01 00 00 01 |......JFIF......| 00000010 00 01 00 00 ff e1 00 2f 45 78 69 66 00 00 49 49 |......./Exif..II| 00000020 2a 00 08 00 00 00 01 00 0e 01 02 00 0d 00 00 00 |*...............| 00000030 1a 00 00 00 00 00 00 00 6b 69 6e 6f 70 6f 69 73 |........kinopois| 00000040 6b 2e 72 75 00 ff fe 00 3b 43 52 45 41 54 4f 52 |k.ru....;CREATOR| 00000050 3a 20 67 64 2d 6a 70 65 67 20 76 31 2e 30 20 28 |: gd-jpeg v1.0 (| 00000060 75 73 69 6e 67 20 49 4a 47 20 4a 50 45 47 20 76 |using IJG JPEG v| 00000070 38 30 29 2c 20 71 75 61 6c 69 74 79 20 3d 20 39 |80), quality = 9| 00000080 31 0a ff db 00 43 00 03 02 02 03 02 02 03 03 02 |1....C..........| 00000090 03 03 03 03 03 04 07 05 04 04 04 04 09 06 07 05 |................| 000000a0 07 0a 09 0b 0b 0a 09 0a 0a 0c 0d 11 0e 0c 0c 10 |................| 000000b0 0c 0a 0a 0e 14 0f 10 11 12 13 13 13 0b 0e 14 16 |................| ...
Go Reader.Discard — skipping bytes in network protocols
The Discard method skips the next n bytes and
discards them.
func (b *Reader) Discard(n int) (discarded int, err error)
This is particularly useful in network protocols where messages have headers, padding, or message types that we want to skip. Instead of reading the data into a buffer and ignoring it, we simply discard it.
In the following example, we simulate a simple protocol where each message
consists of a 4-byte message type, a 4-byte payload length, and the payload
itself. If the message type is a heartbeat (HB), we discard
the payload. Otherwise we read and process it.
package main
import (
"bufio"
"bytes"
"encoding/binary"
"fmt"
"log"
)
func main() {
// Simulated network data: HB (heartbeat, discard) and MSG (process)
buf := new(bytes.Buffer)
writeMessage(buf, "HB", "ping")
writeMessage(buf, "MSG", "hello, there")
writeMessage(buf, "HB", "keepalive")
writeMessage(buf, "MSG", "goodbye")
r := bufio.NewReader(buf)
for i := 0; i < 4; i++ {
typ := make([]byte, 2)
_, err := r.Read(typ)
if err != nil {
log.Fatal(err)
}
var length int32
binary.Read(r, binary.BigEndian, &length)
if string(typ) == "HB" {
_, err := r.Discard(int(length))
if err != nil {
log.Fatal(err)
}
fmt.Printf("discarded heartbeat (%d bytes)\n", length)
} else {
payload := make([]byte, length)
_, err := r.Read(payload)
if err != nil {
log.Fatal(err)
}
fmt.Printf("received message: %s\n", payload)
}
}
}
func writeMessage(buf *bytes.Buffer, typ string, payload string) {
buf.Write([]byte(typ))
binary.Write(buf, binary.BigEndian, int32(len(payload)))
buf.WriteString(payload)
}
The example simulates a network stream where heartbeat messages are discarded and regular messages are processed.
typ := make([]byte, 2) _, err := r.Read(typ)
We read the 2-byte message type identifier from the stream.
var length int32 binary.Read(r, binary.BigEndian, &length)
We read a 4-byte length prefix using encoding/binary.
_, err := r.Discard(int(length))
For heartbeat messages, we discard the payload bytes instead of reading them.
The Discard method advances the reader position by n
bytes without allocating a buffer for the skipped data.
$ go run main.go discarded heartbeat (4 bytes) received message: hello, there discarded heartbeat (9 bytes) received message: goodbye
Go ReadWriter — buffered reader and writer for network connections
The bufio.ReadWriter struct combines a bufio.Reader
and a bufio.Writer into a single type. It is typically used with
network connections, where you need both buffered reading and writing on the
same socket.
type ReadWriter struct {
*Reader
*Writer
}
func NewReadWriter(r *Reader, w *Writer) *ReadWriter
NewReadWriter takes an existing reader and writer pair and
bundles them together. The ReadWriter embeds both types, so all
Reader and Writer methods are available directly on
the ReadWriter value.
In the following example, we build a simple echo client that connects to a
TCP server, sends a message, and reads the echoed response — all through a
single ReadWriter.
package main
import (
"bufio"
"fmt"
"log"
"net"
)
func main() {
ln, err := net.Listen("tcp", ":9999")
if err != nil {
log.Fatal(err)
}
defer ln.Close()
fmt.Println("echo server listening on :9999")
for {
conn, err := ln.Accept()
if err != nil {
log.Fatal(err)
}
go handleConn(conn)
}
}
func handleConn(conn net.Conn) {
defer conn.Close()
rw := bufio.NewReadWriter(
bufio.NewReader(conn), bufio.NewWriter(conn))
for i := 0; i < 3; i++ {
msg, err := rw.ReadString('\n')
if err != nil {
log.Print(err)
return
}
fmt.Printf("received: %s", msg)
rw.WriteString(msg)
rw.Flush()
}
}
The server accepts connections, reads lines, and echoes them back. It uses
NewReadWriter to wrap the connection in both a buffered reader
and a buffered writer.
package main
import (
"bufio"
"fmt"
"log"
"net"
)
func main() {
conn, err := net.Dial("tcp", "localhost:9999")
if err != nil {
log.Fatal(err)
}
defer conn.Close()
rw := bufio.NewReadWriter(
bufio.NewReader(conn), bufio.NewWriter(conn))
messages := []string{"hello\n", "world\n", "bye\n"}
for _, msg := range messages {
rw.WriteString(msg)
rw.Flush()
echo, err := rw.ReadString('\n')
if err != nil {
log.Fatal(err)
}
fmt.Printf("echo: %s", echo)
}
}
The client sends three lines and reads the echoed responses through the same
ReadWriter.
rw := bufio.NewReadWriter(
bufio.NewReader(conn), bufio.NewWriter(conn))
We create a ReadWriter from the connection. The reader and writer
share the same underlying socket.
rw.WriteString(msg) rw.Flush()
We write to the connection through the buffered writer. The Flush
call is essential — without it, the data stays in the buffer.
echo, err := rw.ReadString('\n')
We read the echoed line from the same connection through the buffered reader.
$ go run server.go & echo server listening on :9999 $ go run client.go echo: hello echo: world echo: bye
Performance and buffer sizing
Choosing the right buffer size can have a significant impact on IO performance. Here are the key considerations.
Default buffer size
Both bufio.NewReader and bufio.NewWriter use a
default buffer size of 4096 bytes (4 KB). This is not arbitrary — it
matches the common filesystem block size on Linux and other Unix systems,
meaning the kernel already reads and writes data in 4 KB chunks. Matching
this size avoids wasting space in the buffer while still keeping system call
counts low.
Effect of buffer size on performance
The following benchmark reads a 10 MB file using three different buffer sizes to illustrate the difference.
package main
import (
"bufio"
"fmt"
"io"
"log"
"os"
"time"
)
func readWithBufferSize(size int) (int64, error) {
f, err := os.Open("data.bin")
if err != nil {
return 0, err
}
defer f.Close()
r := bufio.NewReaderSize(f, size)
var total int64
buf := make([]byte, size)
for {
n, err := r.Read(buf)
total += int64(n)
if err == io.EOF {
break
}
if err != nil {
return total, err
}
}
return total, nil
}
func main() {
for _, size := range []int{256, 4096, 65536} {
start := time.Now()
total, err := readWithBufferSize(size)
if err != nil {
log.Fatal(err)
}
elapsed := time.Since(start)
fmt.Printf("buffer size %7d B: %s (read %d bytes)\n",
size, elapsed, total)
}
}
The example reads a 10 MB file with buffer sizes of 256 bytes, 4 KB, and 64 KB, measuring the elapsed time for each.
$ go run main.go buffer size 256 B: 48.3ms (read 10485760 bytes) buffer size 4096 B: 12.1ms (read 10485760 bytes) buffer size 65536 B: 10.8ms (read 10485760 bytes)
The 4 KB buffer is roughly 4× faster than 256 bytes, while increasing to 64 KB yields only a small additional gain. The sweet spot for most workloads is between 4 KB and 64 KB.
Guidelines for choosing buffer size
| Scenario | Recommended size | Reason |
|---|---|---|
| General file IO | 4096 (default) | Matches filesystem block size; good balance of speed and memory. |
| Large sequential reads/writes | 32768–65536 | Fewer system calls for large files; marginal returns diminish beyond 64 KB. |
| Network streaming | 4096–16384 | Network MTU is typically 1500 bytes; too large a buffer increases latency without throughput gain. |
| Memory-constrained environments | 1024–2048 | Smaller buffer uses less memory at the cost of more system calls. |
| Interactive or real-time IO | 512–1024 | Smaller buffers reduce latency between data arrival and processing. |
Writer flush strategies
Buffered writers flush automatically when the buffer is full. When the data is smaller than the buffer, the developer controls when the flush happens. The choice of flush strategy depends on the use case:
- Write-only batch jobs — call
Flushonce after all writes are complete. This maximizes batching and minimizes system calls. - Long-running writers — flush periodically (e.g. every 100 records or every second) to avoid losing large amounts of data on a crash.
- Network connections — flush after each logical message so the remote end receives it promptly, even if the message is small.
Scanner token size limit
By default, bufio.Scanner refuses tokens longer than 65536 bytes
(64 KB). If a line or token exceeds this limit, the scanner returns an error.
To increase the limit, use the Buffer method:
func (s *Scanner) Buffer(buf []byte, max int)
package main
import (
"bufio"
"fmt"
"log"
"strings"
)
func main() {
data := "A" + strings.Repeat("x", 100_000) + "B"
sc := bufio.NewScanner(strings.NewReader(data))
// Increase the buffer to handle 100 KB tokens
sc.Buffer(make([]byte, 100_000), 100_000)
for sc.Scan() {
fmt.Printf("token length: %d\n", len(sc.Text()))
}
if err := sc.Err(); err != nil {
log.Fatal(err)
}
}
The example reads a token longer than the default 64 KB limit by providing a custom buffer and a larger maximum token size.
$ go run main.go token length: 100001
Source
In this article we have worked with the bufio package in Go.
Author
List all Go tutorials.