Brad H February 2016

content from a website in a text file

My aim is to get content from a website (for instance a league table from a sports website) and put it in a .txt file so that I can code with a local file.

I have tried multiple lines of code and others examples such as:

// prepare the web page we will be asking for
    HttpWebRequest request = (HttpWebRequest)
        WebRequest.Create("http://www.stackoverflow.com");
// prepare the web page we will be asking for
    HttpWebRequest request = (HttpWebRequest)
        WebRequest.Create("http://www.stackoverflow.com");

    // execute the request
    HttpWebResponse response = (HttpWebResponse)request.GetResponse();

    // we will read data via the response stream
    Stream resStream = response.GetResponseStream();

    string tempString = null;
    int count = 0;
    do
    {
        // fill the buffer with data
        count = resStream.Read(buf, 0, buf.Length);

        // make sure we read some data
            if (count != 0)
        {
        // translate from bytes to ASCII text
        tempString = Encoding.ASCII.GetString(buf, 0, count);

        // continue building the string
        sb.Append(tempString);
        }
    while (count > 0); // any more data to read?
    }

My issue is when trying this, is that the words request and response are underlined in read and all the tokens are invalid. Is there a better method to get content from a website to a .txt file or is there a way to fix the code supplied?

Thanks

Answers


BornToGrill February 2016

is there a way to fix the code supplied?

The code you submitted works for me, make sure you have the proper name spaces defined. In this case : using System.Net;

Or might it be that the duplicate creation of the variable request isn't a typo? If so remove one of the request variables.

Is there a better method to get content from a website to a .txt file

Since you're reading all the content from the site anyway there isn't really a need for the while loop. Instead you can use the ReadToEnd method supplied by the StreamReader.

string siteContent = "";    
using (StreamReader reader = new StreamReader(resStream)) {
   siteContent = reader.ReadToEnd();
}

Also be sure to dispose of the WebResponse, other than that your code should work fine.

Post Status

Asked in February 2016
Viewed 3,628 times
Voted 4
Answered 1 times

Search




Leave an answer