Spiders
Most Recent Entries
DuckDuckGo Not Respecting Robots.txt Directives?
Nov 27, 2012 - 9:07 am 15 — by Barry SchwartzSpiderDuck: The Realtime Twitter Spider
Nov 18, 2011 - 8:30 am 3 — by Barry SchwartzGoogle: GoogleBot China Crawler A Bug
Mar 10, 2011 - 9:39 am 5 — by Barry SchwartzGoogleBot Caught Masking Around As An iPhone
Feb 23, 2011 - 8:34 am 6 — by Barry SchwartzGoogle On Mobile SEO: GoogleBot & URL Structure
Feb 22, 2011 - 9:20 am 46 — by Barry SchwartzHot In Web Design But Not In SEO? Parallax Effects Design
Feb 16, 2011 - 8:54 am 18 — by Barry SchwartzGoogle Doesn't Use Toolbar Data To Index New URLs
Feb 2, 2011 - 9:11 am 9 — by Barry SchwartzOlder Entries
Google: Remove The Robots.txt File Completely
Jan 6, 2011Google: Canonical Tag Slower Than 301 Redirect
Dec 8, 2010Structure Is King With SEO, Not Content
Dec 6, 2010Google Published Detailed Specifications on Robots.txt
Nov 26, 2010Google Fixes Blogspot Indexing Issue: New Content Being Indexed
Oct 25, 2010Google Stop Indexing Blogger (Blogspot) Posts
Oct 22, 2010Why Google May Not Find Your Sitemap File
Oct 11, 2010Google Changes Coming Tomorrow: AJAX Results, 30 Results Per Page, Streaming Results & More?
Sep 7, 2010Microsoft Cloaking MSNBots Again?
Sep 3, 2010GoogleBot Crawling Less or More Efficiently?
Sep 2, 2010GoogleBot Crawling From Different Locations At Same Time
Aug 24, 2010Google: The Robots Meta Tag Is Cumulative (First Come First Serve)
Jul 30, 2010Google Not Sending Referrer Data Again?
Jul 13, 2010GoogleBot Acting Weird
Jun 30, 2010BingBot Is Coming, BingBot Is Coming: Bye MSNBot
Jun 30, 2010GoogleBot Back In Business, Crawling & Crawling
Jun 25, 2010WordPress System Admin Calls Out Google's Evil Spiders
Jun 23, 2010GoogleBot Stop Crawling Your Site On June 15th?
Jun 16, 2010Bing Updates Language Detection After One or Two Crawls
Apr 28, 2010Bing: We Know About Your Images & Choose Not To Show Them
Mar 8, 2010Bing Masking MSNBot As Internet Explorer or a Rogue Bot?
Feb 22, 2010Bing Indexing New Content & Sites Faster?
Feb 17, 2010Popular Categories
Search Video Recaps
Most Recent Articles
Daily Search Forum Recap: December 2, 2025
Dec 2, 2025 - 10:00 am
0
-
by
Barry Schwartz
22 Years Covering The Search Industry
Dec 2, 2025 - 8:01 am
54
-
by
Barry Schwartz
December 2025 Google Webmaster Report
Dec 2, 2025 - 7:51 am
0
-
by
Barry Schwartz
Google Rolls Out Gemini 3 & Nano Banana Pro In AI Mode To More
Dec 2, 2025 - 7:41 am
0
-
by
Barry Schwartz
Google: 100MB Video Won't Have A Noticeable Effect On Your SEO
Dec 2, 2025 - 7:31 am
1
-
by
Barry Schwartz
Google Officially Tests Blending AI Mode Into AI Overviews
Dec 2, 2025 - 7:21 am
1
-
by
Barry Schwartz
