Spiders

Older Entries

MSNBot Crawl Delay Doesn't Delay

Nov 11, 2009

Bing's MSNBot Crawl Happy?

Oct 23, 2009

Fetch As Googlebot Might Have Issues With Some Non-ASCII Characters

Oct 21, 2009

Fetch As Googlebot Shows Only 100Kb But Real Googlebot Gets More

Oct 14, 2009

Google Labs For Webmaster Tools Launches: Fetch as Googlebot & Malware Details

Oct 13, 2009

Bing Masking MSNBot Under Mozilla's UserAgent & Reverse IP Fails

Sep 17, 2009

Stop Spiders From Crawling Your Site on Shabbat, Including GoogleBot

Sep 9, 2009

Bing Not Honoring Robots.txt Directives?

Sep 9, 2009

Google Crawls Robots.txt Files Daily

Sep 3, 2009

Microsoft Admits To Clicking On Their Own Search Ads

Aug 13, 2009

DigitalPoint Founder Upset With MSNBot's Crawl Rate (MSNBot 2.0b)

Aug 3, 2009

MSNBot Clicking on Bing adCenter Search Ads?

Jul 29, 2009

Microsoft's Search Bot MSNBot 2 Adding Pound Signs to URLs?

Jul 20, 2009

New MSNBot Named adidxbot Causing Trouble

Jul 7, 2009

Tracking Search Bot Activity

Apr 6, 2009

Multiple Robots.txt Files for Single Domain

Apr 1, 2009

Live Search Begins Crawling JavaScript with MSNBot-Media

Jan 2, 2009

Share Your Funny SEO Mistakes

Dec 18, 2008

Stop GoogleBot From Indexing You At Busy Times

Dec 12, 2008

Google's Video Crawler Not Respecting Robots.txt Directives?

Dec 1, 2008

Ditched GoogleBot But Now Want To Make Friends Again?

Nov 14, 2008

CSF (ConfigServer Security and Firewall) May Be Blocking GoogleBot

Nov 3, 2008

Google Webmaster Tools Slower In Reporting Than Google Cache

Sep 24, 2008

How To Tame GoogleBot

Aug 28, 2008

Did Google Stop Crawling Blogger Blogs on Personalized Domains?

Aug 7, 2008

Did Ask.com Stop or Slow Crawling the Web?

Jul 18, 2008

Google Improves Flash Indexing Again

Jul 15, 2008

Google Retraction: Blocking Regions Is Not Cloaking

Jul 3, 2008

Updated: Google Says Blocking Countries Outside of the US is Against Policies

Jul 2, 2008

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Google Core Update Rumbling, Manual Actions FAQs, Core Web Vitals Updates, AI, Bing, Ads & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Google Updates

Google Urges Patience As The March 2024 Core Update Continues To Rollout

Mar 18, 2024 - 7:51 am
Google

Official: Google Replaces Perspective Filter With Forums Filter

Mar 18, 2024 - 7:41 am
Google Maps

Google Business Profiles Now Offers Additional Review After Appeal Is Denied

Mar 18, 2024 - 7:31 am
Google Maps

EU Searchers Complaining About Google Maps Features Changes Related To DMA

Mar 18, 2024 - 7:21 am
Google

Google Showing Fewer Sitelinks Within Search

Mar 18, 2024 - 7:11 am
Search Forum Recap

Daily Search Forum Recap: March 15, 2024

Mar 15, 2024 - 4:00 pm