Spiders
Most Recent Entries
Bing Crawl Consumption Not Showing Return On Investment
May 31, 2018 - 7:49 am 3 — by Barry SchwartzDuckDuckGo Not Respecting Robots.txt Directives?
Nov 27, 2012 - 9:07 am 15 — by Barry SchwartzSpiderDuck: The Realtime Twitter Spider
Nov 18, 2011 - 8:30 am 3 — by Barry SchwartzGoogle: GoogleBot China Crawler A Bug
Mar 10, 2011 - 9:39 am 5 — by Barry SchwartzGoogleBot Caught Masking Around As An iPhone
Feb 23, 2011 - 8:34 am 6 — by Barry SchwartzGoogle On Mobile SEO: GoogleBot & URL Structure
Feb 22, 2011 - 9:20 am 46 — by Barry SchwartzHot In Web Design But Not In SEO? Parallax Effects Design
Feb 16, 2011 - 8:54 am 18 — by Barry SchwartzOlder Entries
Google Doesn't Use Toolbar Data To Index New URLs
Feb 2, 2011Google: Remove The Robots.txt File Completely
Jan 6, 2011Google: Canonical Tag Slower Than 301 Redirect
Dec 8, 2010Structure Is King With SEO, Not Content
Dec 6, 2010Google Published Detailed Specifications on Robots.txt
Nov 26, 2010Google Fixes Blogspot Indexing Issue: New Content Being Indexed
Oct 25, 2010Google Stop Indexing Blogger (Blogspot) Posts
Oct 22, 2010Why Google May Not Find Your Sitemap File
Oct 11, 2010Google Changes Coming Tomorrow: AJAX Results, 30 Results Per Page, Streaming Results & More?
Sep 7, 2010Microsoft Cloaking MSNBots Again?
Sep 3, 2010GoogleBot Crawling Less or More Efficiently?
Sep 2, 2010GoogleBot Crawling From Different Locations At Same Time
Aug 24, 2010Google: The Robots Meta Tag Is Cumulative (First Come First Serve)
Jul 30, 2010Google Not Sending Referrer Data Again?
Jul 13, 2010GoogleBot Acting Weird
Jun 30, 2010BingBot Is Coming, BingBot Is Coming: Bye MSNBot
Jun 30, 2010GoogleBot Back In Business, Crawling & Crawling
Jun 25, 2010WordPress System Admin Calls Out Google's Evil Spiders
Jun 23, 2010GoogleBot Stop Crawling Your Site On June 15th?
Jun 16, 2010Bing Updates Language Detection After One or Two Crawls
Apr 28, 2010Bing: We Know About Your Images & Choose Not To Show Them
Mar 8, 2010Bing Masking MSNBot As Internet Explorer or a Rogue Bot?
Feb 22, 2010Popular Categories
Search Video Recaps
Most Recent Articles
Google Search Broke All Usage Records With Highest Usage Yesterday
Jul 8, 2026 - 11:58 am
6
-
by
Barry Schwartz
Daily Search Forum Recap: July 8, 2026
Jul 8, 2026 - 10:00 am
1
-
by
Barry Schwartz
Google Tests google.com/goto Tracking URLs In Search Results
Jul 8, 2026 - 7:51 am
1
-
by
Barry Schwartz
Google Ads Conversions Last Event Ping Timestamp
Jul 8, 2026 - 7:41 am
0
-
by
Barry Schwartz
Google Ads New AI Max FAQs
Jul 8, 2026 - 7:31 am
0
-
by
Barry Schwartz
Google Search Product Results Contact Store For Price Label
Jul 8, 2026 - 7:21 am
0
-
by
Barry Schwartz
