Older entries:

• Bing Masking MSNBot Under Mozilla's UserAgent & Reverse IP Fails | September 17, 2009
• Stop Spiders From Crawling Your Site on Shabbat, Including GoogleBot | September 9, 2009
• DigitalPoint Founder Upset With MSNBot's Crawl Rate (MSNBot 2.0b) | August 3, 2009
• MSNBot Clicking on Bing adCenter Search Ads? | July 29, 2009
• New MSNBot Named adidxbot Causing Trouble | July 7, 2009
• Tracking Search Bot Activity | April 6, 2009
• Multiple Robots.txt Files for Single Domain | April 1, 2009
• Live Search Begins Crawling JavaScript with MSNBot-Media | January 2, 2009
• Did Google Stop Crawling Blogger Blogs on Personalized Domains? | August 7, 2008
• Did Ask.com Stop or Slow Crawling the Web? | July 18, 2008
• Google Improves Flash Indexing Again | July 15, 2008
• Google Retraction: Blocking Regions Is Not Cloaking | July 3, 2008
• Updated: Google Says Blocking Countries Outside of the US is Against Policies | July 2, 2008
• Can You Remove a Site From a Country Specific Google Search Engine? | June 20, 2008
• Can Google's First Click Free Program be Used for Web Search? | June 5, 2008
• Google Now Crawling Content Behind Forms | April 14, 2008
• Is Google's AdWords Spider Lowercasing Destination URLs? | April 2, 2008
• Yahoo Slurp, Yahoo Search Crawler, Suffering From ADHD? | March 31, 2008
• MSNBot Again Failing Reverse DNS Test | March 11, 2008
• Google's Remove URLs Feature Removes For 90 Days Only | January 22, 2008
• Wordpress Installation Now Blocking Search Engines? | January 22, 2008
• MSNBot Reverse DNS Test Fails Requirements | December 5, 2007
• Does Googlebot Index Faster When Integrating Google Custom Search? | July 16, 2007
• Verify The Bots Accessing Your Site: Is Google.com Sending That GoogleBot? | March 7, 2007
• Google Terminates AdWords Account for "Cookie Spidering"? | November 29, 2006
• Scrape Bots Vs. Search Bots :: Fighting the Battle | September 12, 2006