SBista February 2016

How to open web browser in a loop and enter data and wait until it fully loads

I have a C# form with a web browser control on it. I want to open a url eg:(www. google.com)in a loop and for each time the loop runs I want to first navigate to the url fill a search string and click the search button and wait until the search results load fully.

How can I do this?

I wrote this code to save the url that we get after search result loads but only the search result for the last string seems to load and gets saved in my list.

private void button1_Click(object sender, EventArgs e)
{
    var task = DoNavigationAsync();
    task.ContinueWith((t) =>
    {
        MessageBox.Show("Done!");
    }, TaskScheduler.FromCurrentSynchronizationContext());
}

private void webBrowser1_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
    HtmlElement url = webBrowser1.Document.GetElementById("sb_form_q");

    if (url != null)
    {
        url.SetAttribute("value", search[searchindx-1]);
        webBrowser1.Document.GetElementById("sb_form_go").InvokeMember("click");
    }

    if (webBrowser1.Url.ToString() != "http://www.bing.com/")
    {
        SavedUrl.Add(webBrowser1.Url.ToString());
    }
}

async Task DoNavigationAsync()
{
    TaskCompletionSource<bool> tcsNavigation = null;
    TaskCompletionSource<bool> tcsDocument = null;

    this.webBrowser1.Navigated += (s, e) =>
    {
        if (tcsNavigation.Task.IsCompleted)
            return;
        tcsNavigation.SetResult(true);
    };

    this.webBrowser1.DocumentCompleted += (s, e) =>
    {
        if (this.webBrowser1.ReadyState != WebBrowserReadyState.Complete)
            return;
        if (tcsDocument.Task.IsCompleted)
            return;
        tcsDocument.SetResult(true);
    };
    search = new string[3];
    search[0] = "C";
    search[1] = "C++";
    search[2] = "C#";

    searchindx = 0;

    foreach (string sval in search)
    {
        searchindx++;
        tcsNavigation = new TaskCompletionSource<bool>();
          

Answers


Franz Wimmer February 2016

Using the async HttpClient from .NET Framework 4.5, you can load a web page without using a gui element such as WebBrowser.

A download would look like this:

using (HttpClient client = new HttpClient()) {
    await client.GetStringAsync("https://google.com");
}

This would get you the HTML content of the google search site.

But if you just want to have the resulting URL, you won't even need to perform a download because google (and most other search engines) provides an API for that. Note the following google URL: https://www.google.com/search?q=google. You can see that the search string "google" appears as parameter named "q". So if you build your code like this...

string[] search = new string[] { "C", "C++", "C#" };

foreach (string sval in search)
{
    // C# <= 5
    SavedUrl.Add(string.Format("https://google.com/search?q={0}", sval));
    // C# 6
    SavedUrl.Add($"https://google.com/search?q={sval}");
}

... you won't need any web access.

Post Status

Asked in February 2016
Viewed 3,665 times
Voted 4
Answered 1 times

Search




Leave an answer