Over the past few weeks, I have been noticing threads pop up in the Bing forums with complaints from webmasters that Bing's bot, aka MSNBot is not honoring their robots.txt directives.
It was not just one thread, but at least four. They include one from yesterday over here, one started on September 3rd over here, one from September 1 over here and one started on August 25th over here.
I ignored the first three, trying to give Microsoft the benefit of the doubt. Hoping it is a webmaster issue or someone spoofing MSNBot. But four threads on the same topic, all within a few weeks of each other does stand out as a possible issue.
I have personally not confirmed the issue, since I have no interest in blocking MSNBot from crawling any parts of my sites - but others don't like Bing as much as I do.
Forum discussion at Bing Forums here, here, here and over here.
