user3790827 February 2016

How does Google render it's results on the Search engine results page

I've first noticed that the results are not in the HTML received from the server therefore I opened Dev Tools and started looking at the other files that my browser downloaded.

I've noticed that there was only one xhr request named gcosuc and no json( nor xml or other data files downloaded) therefore I thought the code was embedded in the JS itself.

I then searched all the .js files downloaded by chrome and still I could not find where the search results came from.

The I thought that the search results might be inside and iframe element and because of this they are not shown in Dev Tools. With this hypothesis in mind I looked at the HTML generated by the JS after the DOM was loaded thinking that the result might be embedded in an iframe and again I was wrong.

Does anyone have any idea how Google gets and renders it's search results?

Answers


Dave Bush February 2016

As well as I've been able to figure out, it is all done using JavaScript and rendered dynamically as you scroll.

Google+ works similarly. Only what you see on the screen is actually on the DOM and it all disappears as you scroll.

If you are trying to screen scrape the results, I would recommend rendering into a browser and looking at the resulting DOM. I use Chromium.


Joaquín O February 2016

Made a quick try, and it's indeed an AJAX request. The URI is /s and it passes GET parameters, including what you've search.

The response is some kind of awkward JSON that includes text, styles, and some JS. It's like JSON objects, but separated by this chars: /*""*/

Here's an example (word wrapped because lines are really long):

Example response

Edit I add the DevTools requests made by Google:

XHR Requests

Post Status

Asked in February 2016
Viewed 1,920 times
Voted 8
Answered 2 times

Search




Leave an answer