Google Indexing Content Via Referral Logs?

Apr 21, 2010 - 8:14 am 3 by

Brett Tabke, founder of WebmasterWorld, asked in a WebmasterWorld thread if Google is once again crawling and indexing new content discovered from referral logs? Let me quote his full question:

Google started indexing SERPS from our new site search engine. The question comes up, HOW are they getting those search links? These are full keyword searches as performed by users.

When someone searches Google/bing/yahoo/alltheweb and then comes to WebmasterWorld - we highlight the page with those kw's and we also print "try this search on webmasterworld" with a full keyword link to perform a search on our site search engine. So a raw http link is there on the screen, only when someone kicks out a referral from a search engine. How does that link get into GoogleBot?

The only way I can think of is if google is reading pages via the tool bar or via that google accelerator proxy? Or is this just a reconfirm that Google is crawling out of it's referral logs?

You can see that Google has indexed thousands of search results from the new WebmasterWorld search. Now, Brett just changed the url from newsearch.cgi to search.cgi.

What is interesting is that just yesterday I noticed that thousands of rustysearch results are now in Google. I believe this is new and I will try to get them out but it is an interesting coincidence. More details on RustySearch over here.

In any event, I am not exactly sure what is going on. That is why we have forums to discuss it.

Forum discussion at WebmasterWorld.


Popular Categories

The Pulse of the search community


Search Video Recaps

Google AI Overviews, Ranking Volatility, Web Filter, Google Ads AI Summaries & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: May 17, 2024

May 17, 2024 - 4:00 pm
Search Video Recaps

Search News Buzz Video Recap: Google AI Overviews, Ranking Volatility, Web Filter, Google Ads AI Summaries & More

May 17, 2024 - 8:01 am
Google Search Engine Optimization

Remove Your Content From Google's AI Overviews

May 17, 2024 - 7:51 am
Google Ads

Google Ads AI Summaries Live For Some Advertisers

May 17, 2024 - 7:41 am
Google Maps

Order with Google For Food Delivery Going Away End Of June

May 17, 2024 - 7:31 am
Google Search Engine Optimization

Two New Googlebots: GoogleOther-Image & GoogleOther-Video

May 17, 2024 - 7:21 am
Previous Story: Google Crawls Sitemap File Based On Update Frequency