The folks over at WebmasterWorld have just started noticing Microsoft's new spider arriving. As expected, the new spider is named BingBot.
BingBot will still obey all the rules of its predecessor, MSNBot, but now has a fancy new name.
The original announcement said this would happen on October 1, 2010, so we are a day or two early. Here is one sample find in a webmaster's log files:
207.46.195.227 Tue Sep 28 20:24:15 2010 "GET /widgets.html HTTP/1.1" 200 4321 "-" "Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm)" Connection="Keep-Alive" Accept="*/*" Accept-Encoding="gzip, deflate"
As I said back then... Microsoft said that BingBot will continue to respect and honor the msnbot directives in the robots.txt file, so you do not need to update your robots.txt files yet. But if you are doing more advanced checking, you may need to update your personal scripts. It is also to note that if Microsoft "detect separate sets of directives for bingbot and for any of the older versions of Microsoft search bots (such as msnbot) or a set of directives for all crawlers, the directives for bingbot will take precedence."
Forum discussion at WebmasterWorld.
