April 27, 2006
Google Has a New Crawl Pattern - "New Crawl Priorities"
Brett Tabke has split out a thread that has been getting a lot of attention named Pages dropping out of the index. GoogleGuy replied to that thread stating;
One thing to bear in mind is that Bigdaddy will have different crawl priorities. That can account for some of it. If you've run into any spam problems in the past, you might also want to do a reinclusion request. Otherwise, please send an email to bostonpubcon2006 at gmail.com with the subject line "crawlpages" (all one word), and I'll ask someone to see if they notice any commonalities.
This post, deserves its own thread, so Brett split it out and named it With Googles New Big Daddy Index, Crawling is Changing .
Currently we don't have much more information, than what GoogleGuy posted above. Does "different crawl priorities" relate to Google's new Crawl Caching Proxy? Or is it cache independent? I believe it has to be cache independent, due to the volume of posts in the original thread.
Forum discussion on GoogleGuy's comments at WebmasterWorld.
Like The Story? Vote For It On Yahoo Buzz! Or On Sphinn!
rustybrick in Google Optimization at April 27, 2006 7:48 AM
Comments (1)
Comments
When we were all bally-hooing about "FreshRank" last year, in relation to one of the Google patent applications, none of us ever considered that might have developed the technology for their crawl caching.
It appears to me, however, that it ain't working as it should. An awful lot of forum discussions have sprung up across the Web where people are complaining about lack of indexing, existing content dropping from the index, and what I shall describe as "inequities in crawling" (not directly related to whether a page is found in the cache).
Posted by Michael Martinez at May 1, 2006 11:43