Last December, we reported that MSNBot was failing a reverse DNS lookup. Well, guess what folks - MSNBot is failing again on some IP addresses.
An updated WebmasterWorld thread brought this to my attention and I verified it myself. Here is a sample crawl of MSNBot 1.1, the new MSNBot, crawling under the IP address 65.55.232.12 and the reverse lookup propagates to a host name of bl1sch2041303.phx.gbl with no A-Record for bl1sch2041303.phx.gbl on the domain phx.gbl.
The thread has other IP addresses with MSNBot acting stealthy: 65.55.104.29 - - [29/Feb/2008:02:46:46 +0100] "GET / HTTP/1.1" 403 *** "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)" 65.55.104.173 - - [07/Mar/2008:03:53:54 +0100] "GET / HTTP/1.1" 403 *** "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)" 65.54.165.47 - - [08/Mar/2008:14:05:33 +0100] "GET /robots.txt HTTP/1.1" 200 *** "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)"
The one above is mine, but there are other MSNBot crawls that do pass the reverse DNS test, just some are not.
Forum discussion at WebmasterWorld.
