Older entries:

• The IRS Penalizes Bing? | April 28, 2014
• Blocking Apple Devices On Your Web Site? You May Block Google Also! | April 16, 2014
• Google: Can't Crawl Your Robots.txt Then We Stop Crawling Your Site | January 3, 2014
• Video Recap of Weekly Search Buzz :: February 1, 2013 | February 1, 2013
• DuckDuckGo Not Respecting Robots.txt Directives? | November 27, 2012
• Google: Robots.txt Files Must Be Smaller Than 500KB | January 30, 2012
• BingBot IP Ranges Updated | November 16, 2011
• Blocking Google Video Bot Wont Stop Your Pages Showing In Google Video | December 15, 2010
• Bing's MSNBot Crawls Twice, Once For Compressed HTML & Again For Uncompressed | December 16, 2009
• MSNBot Crawl Delay Doesn't Delay | November 11, 2009
• Bing's MSNBot Crawl Happy? | October 23, 2009
• Bing Masking MSNBot Under Mozilla's UserAgent & Reverse IP Fails | September 17, 2009
• Stop Spiders From Crawling Your Site on Shabbat, Including GoogleBot | September 9, 2009
• DigitalPoint Founder Upset With MSNBot's Crawl Rate (MSNBot 2.0b) | August 3, 2009
• MSNBot Clicking on Bing adCenter Search Ads? | July 29, 2009
• New MSNBot Named adidxbot Causing Trouble | July 7, 2009
• Tracking Search Bot Activity | April 6, 2009
• Multiple Robots.txt Files for Single Domain | April 1, 2009
• Live Search Begins Crawling JavaScript with MSNBot-Media | January 2, 2009
• Did Google Stop Crawling Blogger Blogs on Personalized Domains? | August 7, 2008
• Did Ask.com Stop or Slow Crawling the Web? | July 18, 2008
• Google Improves Flash Indexing Again | July 15, 2008
• Google Retraction: Blocking Regions Is Not Cloaking | July 3, 2008
• Updated: Google Says Blocking Countries Outside of the US is Against Policies | July 2, 2008
• Can You Remove a Site From a Country Specific Google Search Engine? | June 20, 2008
• Can Google's First Click Free Program be Used for Web Search? | June 5, 2008
• Google Now Crawling Content Behind Forms | April 14, 2008
• Is Google's AdWords Spider Lowercasing Destination URLs? | April 2, 2008
• Yahoo Slurp, Yahoo Search Crawler, Suffering From ADHD? | March 31, 2008
• MSNBot Again Failing Reverse DNS Test | March 11, 2008
• Google's Remove URLs Feature Removes For 90 Days Only | January 22, 2008
• Wordpress Installation Now Blocking Search Engines? | January 22, 2008
• MSNBot Reverse DNS Test Fails Requirements | December 5, 2007
• Does Googlebot Index Faster When Integrating Google Custom Search? | July 16, 2007
• Verify The Bots Accessing Your Site: Is Google.com Sending That GoogleBot? | March 7, 2007
• Google Terminates AdWords Account for "Cookie Spidering"? | November 29, 2006
• Scrape Bots Vs. Search Bots :: Fighting the Battle | September 12, 2006