ZetCode

Ruby Net::HTTP tutorial

last modified October 18, 2023

In this article we show how to work with the standard Ruby Net::HTTP module. We grab data, post data, work with JSON, and connect to a secure web page. The tutorial uses Sinatra applications for several examples. Zetcode has also a concise Ruby tutorial.

The Hypertext Transfer Protocol (HTTP) is an application protocol for distributed, collaborative, hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web.

Ruby Net::HTTP provides a rich library which can be used to build HTTP clients.

Ruby Net:HTTP status

The code and message methods of the response give its status.

status_code.rb
#!/usr/bin/ruby

require 'net/http'

uri = URI 'http://webcode.me'
res = Net::HTTP.get_response uri
puts res.message
puts res.code

uri = URI 'http://www.example.com/news/'
res = Net::HTTP.get_response uri
puts res.message
puts res.code

uri = URI 'http://www.urbandicionary.com/define.php?term=Dog'
res = Net::HTTP.get_response uri
puts res.message
puts res.code

We perform three HTTP requests with the get_response method and check for the returned status. 

uri = URI 'http://www.example.com/news/'
res = Net::HTTP.get_response uri
puts res.message
puts res.code

The status of the HTTP response is checked with the message and code methods. 

$ ./status_code.rb 
OK
200
Not Found
404
Found
302

200 is a standard response for successful HTTP requests, 404 tells that the requested resource could not be found, and 302 tells that the resource was temporarily redirected.

Ruby Net::HTTP HEAD request

A HEAD request is an HTTP GET request without a message body. The header of a request/response contains metadata, such as HTTP protocol version or content type.

The head method retrieves document headers. The headers consist of fields, including date, server, content type, or last modification time.

head_req.rb
#!/usr/bin/ruby

require 'net/http'

uri = URI "http://www.webcode.me"
http = Net::HTTP.new uri.host, uri.port

res = http.head '/'

puts res['server']
puts res['date']
puts res['last-modified']
puts res['content-type']
puts res['content-length']

The example prints the server, date, last modification time, content type, and content length of the www.something.com web page. 

$ ./http_head.rb 
nginx/1.6.2
Wed, 03 Feb 2021 10:30:30 GMT
Sat, 20 Jul 2019 11:49:25 GMT
text/html
348

Ruby Net::HTTP GET request

The HTTP GET method requests a representation of the specified resource.

get_content.rb
#!/usr/bin/ruby

require 'net/http'

uri = URI 'http://www.webcode.me'

content = Net::HTTP.get uri

puts content

The script grabs the content of the webcode.me web page. The net/http is designed to work closely with the uri module. 

require 'net/http'

This will also require uri so we do not need to require it separately. 

content = Net::HTTP.get uri

The get method generates a GET request to the specified resource. 

$ ./http_get.rb 
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>My html page</title>
</head>
<body>

    <p>
        Today is a beautiful day. We go swimming and fishing.
    </p>
    
    <p>
         Hello there. How are you?
    </p>
    
</body>
</html>

The following program gets a small web page and strips its HTML tags.

strip_tags.rb
#!/usr/bin/ruby

require 'net/http'

uri = URI "http://example.com"

doc = Net::HTTP.get uri

puts doc.gsub %r{</?[^>]+?>}, ''

The script strips the HTML tags of the example web page. 

puts doc.gsub %r{</?[^>]+?>}, ''

A simple regular expression is used to strip the HTML tags.

Ruby Net::HTTP async requests

In the following example, we make multiple asynchronous requests.

async_req.rb
#!/usr/bin/ruby

require 'async'
require 'net/http'
require 'uri'

urls = [
  'http://webcode.me', 
  'https://example.com',
  'http://httpbin.org',
  'https://www.ruby-lang.org'
]

Async do
  urls.each do |url|
    Async do
      res = Net::HTTP.get_response(URI url)
      puts "#{res.message} #{res.code}"
    end
  end
end

We make non-blocking web requests to four sites and print their status codes and messages. 

$ ./async_req.rb 
OK 200
OK 200
OK 200
OK 200

Sinatra

Sinatra is a popular Ruby web application framework. It is easy to install and set up. Some of our examples will also use a Sinatra application. 

$ sudo gem install sinatra
$ sudo gem install thin

We install Sinatra and Thin web server. If Thin is installed, Sinatra automatically chooses Thin over the default WEBrick server.

main.rb
#!/usr/bin/ruby

require 'sinatra'

get '/' do
    "First application"
end

The application reacts to the / route. It sends a simple message back to the client. 

$ ruby main.rb 
== Sinatra (v2.1.0) has taken the stage on 4567 for development with backup from Thin
2021-02-03 11:36:49 +0100 Thin web server (v1.8.0 codename Possessed Pickle)
2021-02-03 11:36:49 +0100 Maximum connections set to 1024
2021-02-03 11:36:49 +0100 Listening on localhost:4567, CTRL+C to stop

The application is started with ruby main.rb command. The Thin server is launched; it listens on 4567 port. 

$ curl localhost:4567
First application

With the curl command line tool, we connect to the server and access the / route. A message appears on the console.

Sinatra get method

The get method issues a GET request to the server. The GET method requests a representation of the specified resource.

main.rb
#!/usr/bin/ruby

require 'sinatra'

get '/greet' do  
    "Hello #{params[:name]}"
end

This is the Sinatra application file. Upon receiving the /greet route, it returns a message containing the name which was sent by the client.

client_get.rb
#!/usr/bin/ruby

require 'net/http'

uri = URI "http://localhost:4567/greet"

params = { :name => 'Peter' }
uri.query = URI.encode_www_form params

puts Net::HTTP.get uri

The script sends a variable with a value to the Sinatra application. The variable is specified directly in the URL. 

params = { :name => 'Peter' }

This is the parameter that we send to the server. 

uri.query = URI.encode_www_form params

We encode the parameter into the URL with the encode_www_form method. 

puts Net::HTTP.get uri

The get method sends a GET request to the server. It returns the response which is printed to the console. 

$ ./client_get.rb 
Hello Peter

This is the output of the example. 

::1 - - [03/Feb/2021:11:51:46 +0100] "GET /greet?name=Peter HTTP/1.1" 200 11 0.0046

In this log of the Thin server we can see that the parameter was encoded into the URL.

We can directly put the parameter into the URL string.

client_get2.rb
#!/usr/bin/ruby

require 'net/http'

uri = URI "http://localhost:4567/greet?name=Peter"

puts Net::HTTP.get uri

This is another way of issuing the GET message; it is essentially the same as the previous example.

Sinatra get user agent

In this section, we specify the name of the user agent.

main.rb
#!/usr/bin/ruby

require 'sinatra'

get '/agent' do
    request.user_agent
end

The Sinatra application returns the user agent sent by the client.

client_agent.rb
#!/usr/bin/ruby

require 'net/http'

uri = URI "http://localhost:4567"
http = Net::HTTP.new uri.host, uri.port

res = http.get '/agent', {'User-Agent' => 'Ruby script'}
puts res.body

This script creates a simple GET request to the Sinatra application. 

res = http.get '/agent', {'User-Agent' => 'Ruby script'}

The user agent is specified in the second parameter of the get method. 

$ ./client_agent.rb 
Ruby script

The server responded with the name of the agent that we have sent with the request.

Sinatra posting a value

The post method dispatches a POST request on the given URL, providing the key/value pairs for the fill-in form content.

main.rb
#!/usr/bin/ruby

require 'sinatra'

post '/target' do
    "Hello #{params[:name]}"
end

The Sinatra application returns a greeting on the /target route. It takes the value from the params hash.

client_post.rb
#!/usr/bin/ruby

require 'net/http'

uri = URI "http://localhost:4567/target"

params = { :name => 'Peter' }
res = Net::HTTP.post_form uri, params

puts res.body

The script sends a request with a name key having Peter value. The POST request is issued with the Net::HTTP.post_form method. 

$ ./client_post.rb 
Hello Peter

::1 - - [03/Feb/2021:12:00:54 +0100] "POST /target HTTP/1.1" 200 11 0.0015

With the POST method, the value is not send in the request URL.

Retrieving definitions from a dictionary

In the following example, we find definitions of a term on the www.dictionary.com. 

$ sudo gem install nokogiri

To parse HTML, we use the nokogiri gem.

get_term.rb
#!/usr/bin/ruby

require 'net/http'
require 'nokogiri'

term = 'cat'
uri = URI "https://www.dictionary.com/browse/#{term}"

res = Net::HTTP.get uri

doc = Nokogiri::HTML res
doc.css("span.one-click-content").map do |node|
    s = node.text.strip!
    s.gsub!(/\s{3,}/, " ") unless (s == nil)
    puts s unless (s == nil)
end

In this script, we find the definitions of the term cat on www.dictionary.com. The Nokogiri::HTML is used to parse the HTML code. 

uri = URI "https://www.dictionary.com/browse/#{term}"

To perform a search, we append the term at the end of the URL. 

doc = Nokogiri::HTML res
doc.css("span.one-click-content").map do |node|
    s = node.text.strip!
    s.gsub!(/\s{3,}/, " ") unless (s == nil)
    puts s unless (s == nil)
end

We parse the content with the Nokogiri::HTML class. We improve the formatting by removing excessive white space.

JSON

JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write and for machines to parse and generate. 

$ sudo gem install json

We have to install json gem if we haven't done so before.

main.rb
require 'sinatra'
require 'json'

get '/example.json' do
    content_type :json
    { :name => 'Jane', :age => 17 }.to_json
end

The Sinatra application sends JSON data. It uses the to_json method to do the job.

parse_json.rb
#!/usr/bin/ruby

require 'net/http'
require 'json'
 
uri = URI 'http://localhost:4567/example.json'
res = Net::HTTP.get uri 

data = JSON.parse res

puts data["name"]
puts data["age"]

The example reads JSON data sent by the Sinatra application. 

$ ./parse_json.rb 
Jane
17

Next, we send JSON data to a Sinatra application from a Ruby script.

main.rb
require 'sinatra'
require 'json'

post '/readjson' do
    data = JSON.parse request.body.read
    "#{data["name"]} is #{data["age"]} years old"
end

This application reads JSON data and sends back a message with the parsed values.

post_json.rb
#!/usr/bin/ruby

require 'net/http'
require 'json'
 
uri = URI 'http://localhost:4567/readjson'

req = Net::HTTP::Post.new uri.path, initheader = {'Content-Type' =>'application/json'}
req.body = {:name => 'Jane', :age => 17}.to_json

res = Net::HTTP.start(uri.hostname, uri.port) do |http|
    http.request req
end

puts res.body

This script sends JSON data to the Sinatra application and reads its response. 

req = Net::HTTP::Post.new uri.path, initheader = {'Content-Type' =>'application/json'}

The 'application/json' content type must be specified in the header of the request. 

$ ./post_json.rb 
Jane is 17 years old

Redirection

Redirection is the process of forwarding one URL to a different URL. The HTTP response status code 302 is used for temporary URL redirection.

main.rb
require 'sinatra'

get "/oldpage" do  
    redirect to("/files/newpage.html"), 302
end

In the Sinatra application, we use the redirect command to redirect to a different location.

public/files/newpage.html
<!DOCTYPE html>
<html>
<head>
<title>New page</title>
</head>
<body>
<p>
This is a new page
</p>
</body>
</html>

This is the newpage.html file located in the public/files subdirectory.

redirect.rb
#!/usr/bin/ruby

require 'net/http'

uri = URI 'http://localhost:4567/oldpage'

res = Net::HTTP.get_response uri
if res.code == "302"
    res = Net::HTTP.get_response URI res.header['location']
end 

puts res.body

This script accesses the old page and follows the redirect. Note that this works for a single redirection. 

res = Net::HTTP.get_response URI res.header['location']

The header's location field contains the address to which the file was redirected. 

$ ./redirect.rb 
<!DOCTYPE html>
<html>
<head>
<title>New page</title>
</head>
<body>
<p>
This is a new page
</p>
</body>
</html>

::1 - - [03/Feb/2021:13:59:52 +0100] "GET /oldpage HTTP/1.1" 302 - 0.0012
::1 - - [03/Feb/2021:13:59:52 +0100] "GET /files/newpage.html HTTP/1.1" 200 112 0.0051

From the log we see that the request was redirected to a new file name. The communication consists of two GET messages.

In this article we have worked with the Ruby net/http module. There are also similar Ruby HTTPClient and Ruby Faraday on ZetCode.

Author

My name is Jan Bodnar and I am a passionate programmer with many years of programming experience. I have been writing programming articles since 2007. So far, I have written over 1400 articles and 8 e-books. I have over eight years of experience in teaching programming.