MSNBot Again Failing Reverse DNS Test
Last December, we reported that MSNBot was failing a reverse DNS lookup. Well, guess what folks - MSNBot is failing again on some IP addresses.
An updated WebmasterWorld thread brought this to my attention and I verified it myself. Here is a sample crawl of MSNBot 1.1, the new MSNBot, crawling under the IP address 65.55.232.12 and the reverse lookup propagates to a host name of bl1sch2041303.phx.gbl with no A-Record for bl1sch2041303.phx.gbl on the domain phx.gbl.
The thread has other IP addresses with MSNBot acting stealthy:
65.55.104.29 - - [29/Feb/2008:02:46:46 +0100] "GET / HTTP/1.1" 403 *** "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
65.55.104.173 - - [07/Mar/2008:03:53:54 +0100] "GET / HTTP/1.1" 403 *** "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
65.54.165.47 - - [08/Mar/2008:14:05:33 +0100] "GET /robots.txt HTTP/1.1" 200 *** "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
The one above is mine, but there are other MSNBot crawls that do pass the reverse DNS test, just some are not.
Forum discussion at WebmasterWorld.
rustybrick in Microsoft MSN Search at March 11, 2008 6:59 AM
Comments (1)

Comments
Any ideas on why MSN would be doing this?
Posted by Jaan Kanellis at March 12, 2008 16:00