Download a web page

This example shows how to download and store the content of a web page in a string. 

/// <summary>
/// Returns the content of a given web adress as string.
/// </summary>
/// <param name="Url">URL of the webpage</param>
/// <returns>Website content</returns>
public string DownloadWebPage(string Url)
{
    // Open a connection
    HttpWebRequest WebRequestObject = (HttpWebRequest)HttpWebRequest.Create(Url);
 
    // You can also specify additional header values like 
    // the user agent or the referer:
    WebRequestObject.UserAgent	= ".NET Framework/2.0";
    WebRequestObject.Referer	= "http://www.example.com/";
 
    // Request response:
    WebResponse Response = WebRequestObject.GetResponse();
 
    // Open data stream:
    Stream WebStream = Response.GetResponseStream();
 
    // Create reader object:
    StreamReader Reader = new StreamReader(WebStream);
 
    // Read the entire stream content:
    string PageContent = Reader.ReadToEnd();
 
    // Cleanup
    Reader.Close();
    WebStream.Close();
    Response.Close();
 
    return PageContent;
}
Snippet Details



string PageContent = DownloadWebPage("http://www.jonasjohn.de/");

Sorry folks, comments have been deactivated for now due to the large amount of spam.

Please try to post your questions or problems on a related programming board, a suitable mailing list, a programming chat-room,
or use a QA website like stackoverflow because I'm usually too busy to answer any mails related
to my code snippets. Therefore please just mail me if you found a serious bug... Thank you!


Older comments:

Matt April 18, 2011 at 06:34
Great script. I modified it to use using statements:
/// <summary>
/// Returns the content of a given web adress as string.
/// from: http://www.jonasjohn.de/snippets/csharp/download-webpage.htm
/// under public domain
/// </summary>
/// <param name="Url">URL of the webpage</param>
/// <returns>Website content</returns>
public string DownloadWebPage(string Url)
{
// Open a connection
HttpWebRequest webRequestObject = (HttpWebRequest)HttpWebRequest.Create(Url);

// You can also specify additional header values like
// the user agent or the referer:
webRequestObject.UserAgent = "Mozilla/5.0 (Windows; U; MSIE 9.0; WIndows NT 9.0; en-US)";
webRequestObject.Referer = "http://www.google.com/";

// Will contain the content of the page
string pageContent = string.Empty;

// Request response:
using (WebResponse response = webRequestObject.GetResponse())
// Open stream and create reader object:
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
// Read the entire stream content:
pageContent = reader.ReadToEnd();
}

return pageContent;
}
Brandon Meyer March 04, 2011 at 21:22
Still a great solution, awesome work!
Yasir December 16, 2010 at 21:03
Works great. Thanks for sharing.
Jonathan Cnan August 08, 2010 at 00:54
A better method would be to use the WebClient.
setup June 23, 2009 at 11:00
but where u store css files and images?
Zed March 12, 2008 at 00:48
I tried changing it so that the file would be saved to disk:
Writer.Write(Reader.ReadToEnd());
Then, I gave a zip file as input.
Something _is_ downloaded, but the ZIP file is corrupt.
Andy November 17, 2007 at 13:53
I have tested your script on the several website and it's working very fine. But when I have tried to fetch the content of the www.pindigital.com then it's not working.

If you can get the content from the www.pindigital.com , then will rate you a good asp.net developer.