Entries from Search Engine Roundtable tagged with 'msnbot'

Last December, we reported that MSNBot was failing a reverse DNS lookup. Well, guess what folks - MSNBot is failing again on some IP addresses. An updated WebmasterWorld thread brought this to my attention and I verified it myself. Here...

Posted on March 11, 2008 6:59 AM

There are two recent threads in different forums complaining about search engine spiders from Microsoft. A WebmasterWorld thread is specifically complaining about MSRbot, Microsoft's research bot, for attempting to crawl pages that don't exist. The other thread is from DigitalPoint...

Posted on February 25, 2008 7:18 AM

The Live Search Blog announced several updated to their crawler. The first is a name change to reflect the upgrade, previously named msnbot/1.0, it is now named msnbot/1.1. The bulk of the changes include the HTTP Compression and Conditional Get...

Posted on February 13, 2008 7:28 AM

Yesterday we reported that MSNBot failed some reverse DNS tests. In short, if you did a reverse DNS lookup on the IP addresses of their spiders, it would not resolve to search.live.com. The big issue with this is that Webmasters...

Posted on December 6, 2007 7:20 AM

A year ago, Microsoft promised to enable Webmasters a method of verifying MSNbot. Way too often, rogue spiders mask themselves as official spiders from Google, Yahoo, Live Search or Ask.com. The search engines have enabled methods to conduct reverse DNS...

Posted on December 5, 2007 7:22 AM

I have been corresponding with Microsoft about the weird spam-like referrals Live Search was sending to hundreds, if not thousands, of web site log files throughout the web. On September 6th, a Microsoft representative confirmed that these were actual tests...

Posted on December 4, 2007 3:00 PM

Microsoft has jumped on the bandwagon for allowing you to verify if the useragent, MSNBot, that is crawling your site, is truly from Microsoft or being spoofed by some content scraper. Google has released information in the past for verifying...

Posted on November 30, 2006 8:17 AM

Sometimes it is hard to find the different supported Meta tags to help control where Live.com (MSNbot) can crawl on your site and what content they should cache. I dug up the resource on MSN's Site Owner Help section that...

Posted on October 6, 2006 7:54 AM

How cute, seriously, MSN has finally given names to their baby crawlers. You know, Google names their crawlers, i.e. GoogleBot, MediaBot, etc... Yahoo has Slurp, etc. Now MSN has named their crawlers. The MSN Shopping bot is msnbot-products. The MSN...

Posted on August 1, 2006 7:36 AM