Spiders
Most Recent Entries
Bing Crawl Consumption Not Showing Return On Investment
May 31, 2018 - 7:49 am 3 — by Barry SchwartzDuckDuckGo Not Respecting Robots.txt Directives?
Nov 27, 2012 - 9:07 am 15 — by Barry SchwartzSpiderDuck: The Realtime Twitter Spider
Nov 18, 2011 - 8:30 am 3 — by Barry SchwartzGoogle: GoogleBot China Crawler A Bug
Mar 10, 2011 - 9:39 am 5 — by Barry SchwartzGoogleBot Caught Masking Around As An iPhone
Feb 23, 2011 - 8:34 am 6 — by Barry SchwartzGoogle On Mobile SEO: GoogleBot & URL Structure
Feb 22, 2011 - 9:20 am 46 — by Barry SchwartzHot In Web Design But Not In SEO? Parallax Effects Design
Feb 16, 2011 - 8:54 am 18 — by Barry SchwartzOlder Entries
Google Doesn't Use Toolbar Data To Index New URLs
Feb 2, 2011Google: Remove The Robots.txt File Completely
Jan 6, 2011Google: Canonical Tag Slower Than 301 Redirect
Dec 8, 2010Structure Is King With SEO, Not Content
Dec 6, 2010Google Published Detailed Specifications on Robots.txt
Nov 26, 2010Google Fixes Blogspot Indexing Issue: New Content Being Indexed
Oct 25, 2010Google Stop Indexing Blogger (Blogspot) Posts
Oct 22, 2010Why Google May Not Find Your Sitemap File
Oct 11, 2010Google Changes Coming Tomorrow: AJAX Results, 30 Results Per Page, Streaming Results & More?
Sep 7, 2010Microsoft Cloaking MSNBots Again?
Sep 3, 2010GoogleBot Crawling Less or More Efficiently?
Sep 2, 2010GoogleBot Crawling From Different Locations At Same Time
Aug 24, 2010Google: The Robots Meta Tag Is Cumulative (First Come First Serve)
Jul 30, 2010Google Not Sending Referrer Data Again?
Jul 13, 2010GoogleBot Acting Weird
Jun 30, 2010BingBot Is Coming, BingBot Is Coming: Bye MSNBot
Jun 30, 2010GoogleBot Back In Business, Crawling & Crawling
Jun 25, 2010WordPress System Admin Calls Out Google's Evil Spiders
Jun 23, 2010GoogleBot Stop Crawling Your Site On June 15th?
Jun 16, 2010Bing Updates Language Detection After One or Two Crawls
Apr 28, 2010Bing: We Know About Your Images & Choose Not To Show Them
Mar 8, 2010Bing Masking MSNBot As Internet Explorer or a Rogue Bot?
Feb 22, 2010Popular Categories
Search Video Recaps
Most Recent Articles
Google AI Mode Links Updated & Adds Contextual Introductions
Dec 11, 2025 - 7:51 am
0
-
by
Barry Schwartz
Google Preferred Sources Now Global & Adds Spotlighting Subscriptions
Dec 11, 2025 - 7:41 am
0
-
by
Barry Schwartz
Google AdSense Offerwall Optimization: Automatic Enrollment
Dec 11, 2025 - 7:31 am
0
-
by
Barry Schwartz
Bing Tests Hide Sponsored Results Button & Ad Grouping, Like Google Ads
Dec 11, 2025 - 7:21 am
0
-
by
Barry Schwartz
Google Web Guide Expands To All Tab Officially (Opt In Still Needed)
Dec 11, 2025 - 7:11 am
0
-
by
Barry Schwartz
Daily Search Forum Recap: December 10, 2025
Dec 10, 2025 - 10:00 am
2
-
by
Barry Schwartz
