Go Pipe
last modified April 11, 2024
In this article we show how to work with pipes in Golang.
Pipe
A pipe is a form of redirection from one process to another process. It is a unidirectional data channel that can be used for interprocess communication.
The io.Pipe function creates a synchronous in-memory pipe. It can
be used to connect code expecting an io.Reader with code expecting
an io.Writer.
$ go version go version go1.22.2 linux/amd64
We use Go version 1.22.2.
Go pipe simple example
The following example demonstrates the usage of the io.Pipe
function.
package main
import (
"fmt"
"io"
"log"
"os"
)
func main() {
r, w := io.Pipe()
go func() {
fmt.Fprint(w, "Hello there\n")
w.Close()
}()
_, err := io.Copy(os.Stdout, r)
if err != nil {
log.Fatal(err)
}
}
In the code example, we create a pipe with io.Pipe. We write to the
pipe's writer in a goroutine and then copy the data from the pipe's reader
to the standard output using io.Copy.
go func() {
fmt.Fprint(w, "Hello there\n")
w.Close()
}()
We write some data to the pipe's writer in a goroutine. Each write to the
PipeWriter blocks until it has satisfied one or more reads
from the PipeReader that fully consume the written data.
$ go run simple.go Hello there
Go cmd StdoutPipe
The StdoutPipe of a command returns a pipe that will be connected
to the command's standard output when the command starts.
package main
import (
"bufio"
"fmt"
"log"
"os"
"os/exec"
)
func main() {
cmd := exec.Command("ping", "webcode.me")
stdout, err := cmd.StdoutPipe()
if err != nil {
log.Fatal(err)
}
cmd.Start()
buf := bufio.NewReader(stdout)
num := 0
for {
line, _, _ := buf.ReadLine()
if num > 3 {
os.Exit(0)
}
num += 1
fmt.Println(string(line))
}
}
In the code example, we launch a ping command and read four lines of
its output.
cmd := exec.Command("ping", "webcode.me")
We create a command which lauches ping to test the availability
of the webcode.me website.
stdout, err := cmd.StdoutPipe()
We get the standard output of the command.
buf := bufio.NewReader(stdout)
A reader from the standard output is created.
for {
line, _, _ := buf.ReadLine()
if num > 3 {
os.Exit(0)
}
num += 1
fmt.Println(string(line))
}
In a for loop, we read four lines and print them to the console.
$ go run pingcmd.go PING webcode.me (46.101.248.126) 56(84) bytes of data. 64 bytes from 46.101.248.126 (46.101.248.126): icmp_seq=1 ttl=54 time=29.7 ms 64 bytes from 46.101.248.126 (46.101.248.126): icmp_seq=2 ttl=54 time=35.9 ms 64 bytes from 46.101.248.126 (46.101.248.126): icmp_seq=3 ttl=54 time=37.4 ms
Go pipe POST JSON data
In the following example, we post JSON data to the https://httpbin.org/post.
package main
import (
"encoding/json"
"fmt"
"io"
"io/ioutil"
"log"
"net/http"
)
type PayLoad struct {
Content string
}
func main() {
r, w := io.Pipe()
go func() {
defer w.Close()
err := json.NewEncoder(w).Encode(&PayLoad{Content: "Hello there!"})
if err != nil {
log.Fatal(err)
}
}()
resp, err := http.Post("https://httpbin.org/post", "application/json", r)
if err != nil {
log.Fatal(err)
}
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatal(err)
}
fmt.Println(string(body))
}
The example posts a JSON payload to the website and reads its body response.
go func() {
defer w.Close()
err := json.NewEncoder(w).Encode(&PayLoad{Content: "Hello there!"})
if err != nil {
log.Fatal(err)
}
}()
We write a JSON payload to the PipeWriter in a goroutine.
resp, err := http.Post("https://httpbin.org/post", "application/json", r)
The http.Post method expects a reader as its third parameter; we
pass a PipeReader there.
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatal(err)
}
fmt.Println(string(body))
Finally, we read the response body and print it.
$ go run post_json.go
{
"args": {},
"data": "{\"Content\":\"Hello there!\"}\n",
"files": {},
"form": {},
"headers": {
...
Go read standard input through pipe
The following example creates a Go program, which reads data passed from standard intput via the pipe.
package main
import (
"bufio"
"fmt"
"io"
"log"
"os"
)
func main() {
nBytes, nChunks := int64(0), int64(0)
r := bufio.NewReader(os.Stdin)
buf := make([]byte, 0, 4*1024)
for {
n, err := r.Read(buf[:cap(buf)])
buf = buf[:n]
if n == 0 {
if err == nil {
continue
}
if err == io.EOF {
break
}
log.Fatal(err)
}
nChunks++
nBytes += int64(len(buf))
fmt.Println(string(buf))
if err != nil && err != io.EOF {
log.Fatal(err)
}
}
fmt.Println("Bytes:", nBytes, "Chunks:", nChunks)
}
The example reads the data from the standard input and prints the data and the number of bytes and chunks read.
r := bufio.NewReader(os.Stdin)
We create a reader from the standard input.
buf := make([]byte, 0, 4*1024)
Here we create a buffer of 4KB.
n, err := r.Read(buf[:cap(buf)]) buf = buf[:n]
We read the data from the standard input into the buffer.
nChunks++ nBytes += int64(len(buf))
We calculate the number of chunks and bytes read.
fmt.Println(string(buf))
We print the contents of the buffer to the terminal.
$ date | go run read_stdin.go Sun 15 Nov 2020 01:08:13 PM CET Bytes: 32 Chunks: 1
We pass the output of the | operator to our program. The Go program
reads the data, counts the number of bytes and prints the data to the console.
Go Stat
The Stat function returns the FileInfo structure describing file.
We can use it to check if there is some data from the pipe on the terminal.
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main() {
stat, _ := os.Stdin.Stat()
if (stat.Mode() & os.ModeCharDevice) == 0 {
var buf []byte
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
buf = append(buf, scanner.Bytes()...)
}
if err := scanner.Err(); err != nil {
log.Fatal(err)
}
fmt.Printf("Hello %s!\n", buf)
} else {
fmt.Print("Enter your name: ")
var name string
fmt.Scanf("%s", &name)
fmt.Printf("Hello %s!\n", name)
}
}
The example takes the input from the user either via the pipe or from the prompt.
stat, _ := os.Stdin.Stat()
We get the FileInfo structure of the standard input.
if (stat.Mode() & os.ModeCharDevice) == 0 {
We check if the input data comes from a pipe.
var buf []byte
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
buf = append(buf, scanner.Bytes()...)
}
We read the data passed via the | pipe operator on the terminal.
} else {
fmt.Print("Enter your name: ")
var name string
fmt.Scanf("%s", &name)
fmt.Printf("Hello %s!\n", name)
}
If there is no data from the pipe, we read the data from the prompt.
$ echo "Peter" | go run hello.go Hello Peter! $ go run hello.go Enter your name: Peter Hello Peter!
First, we pass data to the program via the pipe, then via prompt.
Go pipe in HTTP handler
In the following example, we use a pipe in an HTTP handler.
package main
import (
"fmt"
"io"
"net/http"
"os/exec"
)
func handler(w http.ResponseWriter, r *http.Request) {
cmd := exec.Command("date")
pr, pw := io.Pipe()
defer pw.Close()
cmd.Stdout = pw
cmd.Stderr = pw
go io.Copy(w, pr)
cmd.Run()
}
func main() {
http.HandleFunc("/", handler)
fmt.Println("server started on port 8080")
http.ListenAndServe(":8080", nil)
}
The example starts a simple server, which returns today's date.
cmd := exec.Command("date")
We will execute the date command.
pr, pw := io.Pipe() defer pw.Close()
We create a Go pipe.
cmd.Stdout = pw cmd.Stderr = pw
We pass the PipeWriter to the command's standard output and
standard error output.
go io.Copy(w, pr)
In a goroutine, we copy the contents of the PipeReader to the
http.ResponseWriter.
cmd.Run()
The command is executed with Run.
$ go run handler.go server started on port 8080
We run the server.
$ curl localhost:8080 Sun 15 Nov 2020 02:18:07 PM CET
In a different terminal window, we create a GET request with curl.
Source
In this article we have worked with a pipe in Go.
Author
List all Go tutorials.