(12)
In October, a WebmasterWorld member has been monitoring his website and has noticed that Google is beginning to index pages that seem unlinked from anywhere. He suspects that either Google is using Toolbar data, someone is linking to these pages deliberately, or he may have caused the pages to be linked somehow (but he's pretty sure that's not the case).
Others notice similar "creative" spidering, and as Tedster puts it, "googlebot [is] trying to eat almost anything that might be edible in even the least way."
After investigating further, this is what appears to be happening, according to the member who discovered the issue:
- Googlebot is spidering GET forms by getting the form variables and either leaving them blank or assigning values to them (sometimes taken from options in the form itself) - Google has a list of words present on the site - This list of words is being used to populate the form variables, and the URL requested via GET
It seems that it's now February and the same member found similar strange activity within Google. The site has over 519,000 spider requests from Google since the end of October. He believes that Googlebot is adding the GET data by itself, either by accident or to discover new content. What do you think it is?
Forum discussion continues at WebmasterWorld.
Comments:
Pratheep
02/20/2008 07:46 am
I created a blog for my website and it was liked nowhere from the website, but after a weeks time, the blog got indexed in Google :) with some good rankings. How is that possible? Pratheep
Jeanette
02/20/2008 08:29 am
I uploaded a new website and sent a link to myself in my gmail account. The site was indexed and appeared on a search within a few days. I didn't post the link in my articles or press releases or promote the site in anyway. Joy~ Jeanette
John
02/20/2008 09:23 am
Wordpress blogs are normally indexed even without external links. However I also noticed that there are lots of internal links that probably lead to an already indexed website.
Phil
02/20/2008 12:43 pm
Already seen this, A website published, never linked from any website and visible in Google results. I never submitted the website on any search engine, never added link on any website, nor added adwords on web pages of the website. I didn't found from where Google came in.
Sebastian
02/20/2008 01:52 pm
Folks check your blog's pings. Also think of crawled referrer stats and stuff like that.
James
02/20/2008 04:11 pm
blog engine like wordpress and blogspot will automatic "ping" your new post , it is another form of submit your post (new page) to blog engines, then google spider will crawl these engines and find your blogs.
Jon
02/20/2008 04:21 pm
I have a couple of sites that had their stats pages indexed, and no links to those pages are to be found.
Bill Kruse
02/20/2008 06:44 pm
No reciprocal from the stats host? I'm thinking maybe a link was followed from there. BB
No Name
02/22/2008 07:58 am
Wordpress blogs are normally indexed even without external links. However I also noticed that there are lots of internal links that probably lead to an already indexed website.
IrishWonder
02/22/2008 11:28 pm
Wordpress blogs normally ping at least something when posting, don't they? So no wonder about blogs getting indexed
Amritanshu
09/22/2008 12:20 pm
blogs sites get automatically as you are using a service of an already existing indexed site which will ping on updates let it be interms of a new blog or a new post.. but for a site witohut been linked to anywhere.. need to think over
Tom Studer
08/04/2010 07:05 pm
Following up on Jeanette's comment, we registered a new domain, set up a new website on it (only accessible through a particular path e.g. domain.net/websitepreview/). We never linked to it - neither the domain nor the whole URL. After a while we realized that the website content got Google indexed. Just like Jeanette, we did communicate the preview link in personal emails going through Google Mail.