Ruby Faraday tutorial

last modified January 10, 2023

In this tutorial, we show how to work with the Ruby Faraday module. We grab data, post data, work with JSON, and connect to a secure web page. We also create a custom Faraday middleware. The tutorial uses Sinatra applications for several examples. ZetCode has also a concise Ruby tutorial.

The Hypertext Transfer Protocol (HTTP) is an application protocol for distributed, collaborative, hypermedia information systems. HTTP is the foundation of data communication for the World Wide Web.

Ruby Faraday is a simple, flexible HTTP client library, with support for multiple backends. Faraday also is a middleware.

$ sudo gem install faraday

The module is installed with the sudo gem install faraday command.

Sinatra

Sinatra is a popular Ruby web application framework. It is easy to install and set up. Some of our examples will also use a Sinatra application.

$ sudo gem install sinatra
$ sudo gem install thin

We install Sinatra and Thin web server. If Thin is installed, Sinatra automatically chooses Thin over the default WEBrick server.

$ pwd
/home/janbodnar/prog/sinatra/first
$ ls
main.rb

Inside the first directory, we have a main.rb file, which is the Sinatra application file.

main.rb

require 'sinatra'

get '/' do
    "First application"
end

The application reacts to the / route. It sends a simple message back to the client.

$ ruby main.rb 
== Sinatra (v1.4.7) has taken the stage on 4567 for development with backup from Thin
Thin web server (v1.6.4 codename Gob Bluth)
Maximum connections set to 1024
Listening on localhost:4567, CTRL+C to stop

The application is started with ruby main.rb command. The Thin server is launched; it listens on 4567 port.

$ curl localhost:4567/
First application

With the curl command line tool, we connect to the server and access the / route. A message appears on the console.

Version

The first Faraday program prints the version of the library and of the Ruby language.

version.rb

#!/usr/bin/ruby

require 'faraday'

puts Faraday::VERSION
puts Faraday::default_adapter

The two constants provide the library version number and the default Faraday adapter.

$ ./version.rb 
0.9.2
net_http

This is a sample output of the string.

Getting content

The get method fetches documents identified by the given URL.

get_content.rb

#!/usr/bin/ruby

require 'faraday'

res = Faraday.get 'http://www.something.com' 

puts res.body

The script grabs the content of the www.something.com web page.

$ ./get_content.rb 
<html><head><title>Something.</title></head>
<body>Something.</body>
</html>

This is the output of the get_content.rb script.

The following program gets a small web page and strips its HTML tags.

strip_tags.rb

#!/usr/bin/ruby

require 'faraday'

con = Faraday::Connection.new "http://www.something.com"

res = con.get

puts res.body.gsub(%r{</?[^>]+?>}, '')

The script strips the HTML tags of the www.something.com web page.

puts res.body.gsub(%r{</?[^>]+?>}, '')

A simple regular expression is used to strip the HTML tags.

$ ./strip_tags.rb 
Something.
Something.

The script prints the web page's title and content.

Status

The status method of the Faraday::Response returns the HTTP status code of the response.

status.rb

#!/usr/bin/ruby

require 'faraday'

res = Faraday.get 'http://www.something.com'  
puts res.status
puts res.success?

res = Faraday.get 'http://www.something.com/news/'
puts res.status
puts res.success?

res = Faraday.get 'http://www.urbandicionary.com/define.php?term=Dog'
puts res.status
puts res.success?

We perform three HTTP requests with the get method and check for returned status.

res = Faraday.get 'http://www.something.com'  
puts res.status

The status of the HTTP response is checked with the status method.

puts res.success?

The success? method tells whether the status code was successful.

$ ./status.rb 
200
true
404
false
302
false

200 is a standard response for successful HTTP requests, 404 tells that the requested resource could not be found, and 302 tells that the resource was temporarily redirected.

The head method

The head method retrieves document headers. The headers consist of fields, including date, server, content type, or last modification time.

head.rb

#!/usr/bin/ruby

require 'faraday'

con = Faraday.new :url => "http://www.something.com"

res = con.head 

puts res.headers['server']
puts res.headers['date']
puts res.headers['last-modified']
puts res.headers['content-type']
puts res.headers['content-length']

The example prints the server, date, last modification time, content type, and content length of the www.something.com web page.

$ ./head.rb 
Apache/2.4.12 (FreeBSD) OpenSSL/1.0.1l-freebsd mod_fastcgi/mod_fastcgi-SNAP-0910052141
Tue, 10 May 2016 10:19:01 GMT
Mon, 25 Oct 1999 15:36:02 GMT
text/html
77

This is the output of the head.rb program.

The get method

The get method issues a GET request to the server. The GET method requests a representation of the specified resource.

main.rb

require 'sinatra'

get '/greet' do  
    "Hello #{params[:name]}"
end

This is the Sinatra application file. Upon receiving the /greet route, it returns a message containing the name which was sent by the client.

mget.rb

#!/usr/bin/ruby

require 'faraday'

con = Faraday.new 

res = con.get 'http://localhost:4567/greet', { :name => 'Peter' }

puts res.body

The script sends a variable with a value to the Sinatra application. The variable is specified directly in the URL.

$ ./mget.rb 
Hello Peter

This is the output of the example.

127.0.0.1 - - [10/May/2016:22:04:38 +0200] "GET /greet?name=Peter HTTP/1.1" 200 11 0.0034

In this log of the Thin server we can see that the parameter was encoded into the URL.

The get method takes a second parameter where we can specify the query parameters.

mget2.rb

#!/usr/bin/ruby

require 'faraday'

res = Faraday.get do |req|
    req.url 'http://localhost/greet' 
    req.params['name']  = 'Jan' 
end

puts res.body

This is another way of issuing the GET message.

$ ./mget2.rb 
Hello Peter

This is the output of the example.

User agent

In this section, we specify the name of the user agent.

main.rb

require 'sinatra'

get '/agent' do
    request.user_agent
end

The Sinatra application returns the user agent sent by the client.

agent.rb

#!/usr/bin/ruby

require 'faraday'

con = Faraday.new 

res = con.get do |req| 
    req.url 'http://localhost:4567/agent' 
    req.headers['User-Agent'] = 'Ruby script'
end

puts res.body

This script creates a simple GET request to the Sinatra application.

res = con.get do |req| 
    req.url 'http://localhost:4567/agent' 
    req.headers['User-Agent'] = 'Ruby script'
end

The user agent is specified in the headers attribute of the request.

$ ./agent.rb 
Ruby script

The server responded with the name of the agent that we have sent with the request.

Posting a value

The post method dispatches a POST request on the given URL, providing the key/value pairs for the fill-in form content.

main.rb

require 'sinatra'

post '/target' do
    "Hello #{params[:name]}"
end

The Sinatra application returns a greeting on the /target route. It takes the value from the params hash.

mpost.rb

#!/usr/bin/ruby

require 'faraday'

con = Faraday.new 'http://localhost'

res = con.post '/target', { :name => 'Jan' }

puts res.body

The script sends a request with a name key having Jan value. The POST request is issued with the post method.

$ ./mpost.rb 
Hello Jan

This is the output of the mpost.rb script.

127.0.0.1 - - [11/May/2016:13:49:44 +0200] "POST /target HTTP/1.1" 200 9 0.0006

With the POST method, the value is not send in the request URL.

Retrieving definitions from a dictionary

In the following example, we find definitions of a term on the www.dictionary.com. To parse HTML, we use the nokogiri gem. It can be installed with the sudo gem install nokogiri command.

get_term.rb

#!/usr/bin/ruby

require 'faraday'
require 'nokogiri'

term = 'dog'
con = Faraday.new :url => 'http://www.dictionary.com/browse/'+term

res = con.get 

doc = Nokogiri::HTML res.body
doc.css("div.def-content").map do |node|
    s = node.text.strip!
    s.gsub!(/\s{3,}/, " ") unless (s == nil)
    puts s unless (s == nil)
end

In this script, we find the definitions of the term dog on www.dictionary.com. The Nokogiri::HTML is used to parse the HTML code.

con = Faraday.new :url => 'http://www.dictionary.com/browse/'+term

To perform a search, we append the term at the end of the URL.

doc = Nokogiri::HTML res.body
doc.css("div.def-content").map do |node|
    s = node.text.strip!
    s.gsub!(/\s{3,}/, " ") unless (s == nil)
    puts s unless (s == nil)
end

We parse the content with the Nokogiri::HTML class. The definitions are located inside the <div class="def-content"> tag. We improve the formatting by removing excessive white space.

JSON

JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write and for machines to parse and generate.

$ sudo gem install json

We have to install json gem if we haven't done so before.

main.rb

require 'sinatra'
require 'json'

get '/example.json' do
    content_type :json
    { :name => 'Jane', :age => 17 }.to_json
end

The Sinatra application sends JSON data. It uses the to_json method to do the job.

parse_json.rb

#!/usr/bin/ruby

require 'faraday'
require 'json'
 
con = Faraday.new :url => 'http://localhost:4567/example.json'
  
res = con.get
data = JSON.parse res.body

puts data["name"]
puts data["age"]

The example reads JSON data sent by the Sinatra application.

$ ./parse_json.rb 
Jane
17

This is the output of the example.

Next, we send JSON data to a Sinatra application from a Ruby script.

main.rb

require 'sinatra'
require 'json'

post '/readjson' do
    data = JSON.parse request.body.read
    puts data
    "#{data["name"]} is #{data["age"]} years old"
end

This application reads JSON data and sends back a message with the parsed values.

post_json.rb

#!/usr/bin/ruby

require 'faraday'

con = Faraday.new 

res = con.post do |req|
    req.url 'http://localhost:4567/readjson'
    req.headers['Content-Type'] = 'application/json'
    req.body = '{ "name": "Jane", "age": 17 }'
end

puts res.body

This script sends JSON data to the Sinatra application and reads its response.

req.headers['Content-Type'] = 'application/json'

The 'application/json' content type must be specified in the request.

$ ./post_json.rb 
Jane is 17 years old

This is the output of the example.

Credentials

The basic_auth method sets the name and password to be used for a realm. A security realm is a mechanism used for protecting web application resources.

$ sudo gem install sinatra-basic-auth

For this example, we need to install the sinatra-basic-auth gem.

main.rb

require 'sinatra'
require "sinatra/basic_auth"

authorize do |username, password|
    username == "user7" && password == "7user"
end

get '/' do
    "hello"
end

protect do
    get "/secure" do
        "This is restricted area"
    end
end

In the Sinatra application, we specify the authorization logic and set a protected route.

credentials.rb

#!/usr/bin/ruby

require 'faraday'

con = Faraday.new :url => 'http://localhost/secure/'

user = 'user7'
passwd = '7user'

con.basic_auth  user, passwd
res = con.get 

puts res.body

The script connects to the secure webpage; it provides the user name and the password necessary to access the page.

$ ./credentials.rb 
This is restricted area

With the right credentials, the credentials.rb script returns the restricted data.

Faraday middleware

Middleware is a software that connects two otherwise separate applications. In addition to being an HTTP client, Faraday also acts as a middleware. The concept is very similar to Ruby Rack.

Faraday::Connection contains a list of middlewares. Faraday middlewares are passed an env hash that has request and response information. The middlewares can manipulate this information before and after a request is executed.

Redirection

Redirection is the process of forwarding one URL to a different URL. The HTTP response status code 302 is used for temporary URL redirection.

Redirection is implemented in one of the Faraday middleware modules.

$ sudo gem install faraday_middleware

These modules are available in the faraday_middleware gem.

main.rb

require 'sinatra'

get "/oldpage" do  
    redirect to("/files/newpage.html"), 302
end

In the Sinatra application, we use the redirect command to redirect to a different location.

newpage.html

<!DOCTYPE html>
<html>
<head>
<title>New page</title>
</head>
<body>
<p>
This is a new page
</p>
</body>
</html>

This is the newpage.html file located in the public/files subdirectory.

redirect.rb

#!/usr/bin/ruby

require 'faraday'
require 'faraday_middleware'

con = Faraday.new 'http://localhost:4567/oldpage' do |con|
    con.use FaradayMiddleware::FollowRedirects, limit: 5
    con.adapter Faraday.default_adapter
end

res = con.get
puts res.body

This script accesses the old page and follows the redirect.

$ ./redirect.rb 
<!DOCTYPE html>
<html>
<head>
<title>New page</title>
</head>
<body>
<p>
This is a new page
</p>
</body>
</html>

This is the output of the example.

127.0.0.1 - - [10/May/2016:22:14:16 +0200] "GET /oldpage HTTP/1.1" 302 - 0.0199
127.0.0.1 - - [10/May/2016:22:14:16 +0200] "GET /files/newpage.html HTTP/1.1" 200 113 0.0073

From the log we see that the request was redirected to a new file name. The communication consists of two GET messages.

MyLogger

In the following example, we create our own small middleware. It implements request and response logging.

main.rb

require 'sinatra'

get '/greet' do  
    "Hello #{params[:name]}"
end

This is a Sinatra application which sends a greeting to the client.

logger.rb

#!/usr/bin/ruby

require 'faraday'
require 'logger'

class MyLogger
    
    def initialize app
        @app = app
        @logger = Logger.new(STDOUT)
    end

    def call env
        on_request("request", env)
        @app.call(env).on_complete do
            on_response("response", env)
        end
    end

    private
    def on_request phase, env
        @logger.info("#{phase} : #{env.method} - #{env.url}") if env.method and env.url
    end
    
    private
    def on_response phase, env
        @logger.info("#{phase} : #{env.body}") if env.body 
    end    
end

con = Faraday.new(:url => "http://localhost:4567") do |build|
    build.request :url_encoded
    build.use MyLogger
    build.adapter  Faraday.default_adapter
end

res = con.get "/greet", {'name' => 'Jan'}

Here we create a middleware that implements logging to the console.

def call env
    on_request("request", env)
    @app.call(env).on_complete do
        on_response("response", env)
    end
end

A middleware must implement the call method. It executes a method for a request and for a response.

private
def on_request phase, env
    @logger.info("#{phase} : #{env.method} - #{env.url}") if env.method and env.url
end

Upon generation of a request, the on_request method is called. The method logs the phase, request method, and URL.

con = Faraday.new(:url => "http://localhost:4567") do |build|
    build.request :url_encoded
    build.use MyLogger
    build.adapter Faraday.default_adapter
end

The MyLogger middleware is added to the stack with the use method. When a connection object executes a request, it creates a shared env hash, wraps the outer middlewares around each inner middleware, and executes the call method.

res = con.get "/greet", {'name' => 'Jan'}

A message is sent to the Sinatra application. The request and the response are logged to the terminal.

$ ./logger.rb 
I, [2016-05-11T14:48:55.700198 #4945]  INFO -- : request : get - http://localhost:4567/greet?name=Jan
I, [2016-05-11T14:48:55.706989 #4945]  INFO -- : response : Hello Jan

This is the output of the example.

In this tutorial, we have worked with the Ruby Faraday module. There are similar Ruby HTTPClient tutorial and Ruby Net::HTTP tutorial on ZetCode.

Ebooks

Ruby Faraday tutorial

Sinatra

Version

Getting content

Status

The head method

The get method

User agent

Posting a value

Retrieving definitions from a dictionary

JSON

Credentials

Faraday middleware

Redirection

MyLogger