Reading a web page in C#

last modified October 18, 2026

In this article we show how to get a web page in C#.

The article shows how to read a page using the built-in HttpClient and the third-party Flurl.Http and RestSharp libraries. In the examples, we read the example.com page, which is reserved for use in documentation.

C# read web page with HttpClient

HttpClient provides a class for sending HTTP requests and receiving HTTP responses from a resource identified by a URI. It is the standard way of fetching web pages in .NET.

using var client = new HttpClient();
client.DefaultRequestHeaders.UserAgent.ParseAdd("C# console program");

string content = await client.GetStringAsync("https://example.com");
Console.WriteLine(content);

The code example reads a web page asynchronously with HttpClient.

client.DefaultRequestHeaders.UserAgent.ParseAdd("C# console program");

We set the User-Agent header, which is sent with every request of the client. Some websites reject requests without it.

string content = await client.GetStringAsync("https://example.com");

The GetStringAsync method sends a GET request and reads the body of the response into a string. It throws an HttpRequestException if the response has an error status code. The await operator suspends the program until the operation completes, without blocking the thread.

$ dotnet run
<!doctype html><html lang=en><head><meta charset=utf-8><link rel=icon href=data:,><meta name=viewport content="width=device-width,initial-scale=1"><title>Example Domain</title>...

The page is minified, so the whole document is on one line.

C# HttpClient response details

With GetAsync, we get the whole response, including the status code and the headers.

using var client = new HttpClient();

using HttpResponseMessage resp = await client.GetAsync("https://example.com");

Console.WriteLine($"Status: {(int) resp.StatusCode} {resp.ReasonPhrase}");
Console.WriteLine($"Content type: {resp.Content.Headers.ContentType}");
Console.WriteLine($"Charset: {resp.Content.Headers.ContentType?.CharSet}");

string html = await resp.Content.ReadAsStringAsync();

int start = html.IndexOf("<title>") + "<title>".Length;
int end = html.IndexOf("</title>");

Console.WriteLine($"Title: {html[start..end]}");
Console.WriteLine($"Length: {html.Length} characters");

We print the status, the content type, and the title of the page.

Console.WriteLine($"Content type: {resp.Content.Headers.ContentType}");
Console.WriteLine($"Charset: {resp.Content.Headers.ContentType?.CharSet}");

The Content-Type header contains the media type and the character encoding of the page. The ReadAsStringAsync method uses this encoding to decode the bytes of the body.

int start = html.IndexOf("<title>") + "<title>".Length;
int end = html.IndexOf("</title>");

Console.WriteLine($"Title: {html[start..end]}");

We extract the title with simple string searching. For real HTML processing, an HTML parser should be used; see the AngleSharp article.

$ dotnet run
Status: 200 OK
Content type: text/html; charset=utf-8
Charset: utf-8
Title: Example Domain
Length: 577 characters

C# HttpClient errors

Reading a page can fail for many reasons: the page does not exist, the host name cannot be resolved, or the server does not answer.

using var client = new HttpClient { Timeout = TimeSpan.FromSeconds(10) };

string[] urls =
[
    "https://example.com",
    "https://httpbin.org/status/404",
    "https://nonexistent.invalid"
];

foreach (var url in urls)
{
    try
    {
        string content = await client.GetStringAsync(url);
        Console.WriteLine($"{url}: {content.Length} characters");
    }
    catch (HttpRequestException e) when (e.StatusCode is not null)
    {
        Console.WriteLine($"{url}: HTTP error {(int) e.StatusCode}");
    }
    catch (HttpRequestException e)
    {
        Console.WriteLine($"{url}: {e.HttpRequestError}");
    }
    catch (TaskCanceledException)
    {
        Console.WriteLine($"{url}: timed out");
    }
}

We try to read three URLs.

catch (HttpRequestException e) when (e.StatusCode is not null)
{
    Console.WriteLine($"{url}: HTTP error {(int) e.StatusCode}");
}

If the server returns an error status code, the exception has the StatusCode property set. We handle this case with an exception filter.

catch (HttpRequestException e)
{
    Console.WriteLine($"{url}: {e.HttpRequestError}");
}

Other failures, such as network errors, have no status code. The HttpRequestError property tells us the kind of the error.

catch (TaskCanceledException)
{
    Console.WriteLine($"{url}: timed out");
}

When the Timeout of the client elapses, a TaskCanceledException is thrown.

$ dotnet run
https://example.com: 577 characters
https://httpbin.org/status/404: HTTP error 404
https://nonexistent.invalid: NameResolutionError

C# read web page with Flurl.Http

Flurl.Http is a fluent, portable, testable HTTP client library for .NET.

$ dotnet add package Flurl.Http

We install the Flurl.Http package.

using Flurl.Http;

string result = await "https://example.com"
    .WithHeader("User-Agent", "C# console program")
    .GetStringAsync();

Console.WriteLine(result[..120]);

The example reads the page and prints its first 120 characters.

string result = await "https://example.com"
    .WithHeader("User-Agent", "C# console program")
    .GetStringAsync();

Flurl adds extension methods to strings. We build the request in a fluent way directly from the URL.

$ dotnet run
<!doctype html><html lang=en><head><meta charset=utf-8><link rel=icon href=data:,><meta name=viewport content="width=dev

C# read web page with RestSharp

RestSharp is a simple REST and HTTP API client for .NET. It is a third-party library.

$ dotnet add package RestSharp

We install the RestSharp package.

using RestSharp;

using var client = new RestClient("https://example.com");
var request = new RestRequest();

RestResponse res = await client.ExecuteGetAsync(request);

Console.WriteLine($"{(int) res.StatusCode} {res.ContentType}");
Console.WriteLine(res.Content?[..120]);

The code example gets the contents of a web page using the RestSharp library.

using var client = new RestClient("https://example.com");

A client is created with the RestClient class. It is disposable, because it wraps an HttpClient.

var request = new RestRequest();

A request is created with RestRequest; it is relative to the base URL of the client.

RestResponse res = await client.ExecuteGetAsync(request);

The request is executed with the ExecuteGetAsync method. Unlike GetStringAsync of HttpClient, it does not throw on an error status code; we check the StatusCode or the IsSuccessful property of the response.

$ dotnet run
200 text/html
<!doctype html><html lang=en><head><meta charset=utf-8><link rel=icon href=data:,><meta name=viewport content="width=dev

Source

HttpClient class - language reference

In this article we have shown how to read a web page in C#.

Author

My name is Jan Bodnar, and I am a passionate programmer with extensive programming experience. I have been writing programming articles since 2007. To date, I have authored over 1,400 articles and 8 e-books. I possess more than ten years of experience in teaching programming.

List all C# tutorials.