Bing Not Honoring Robots.txt Directives?

Sep 9, 2009 - 8:45 am 0 by
Filed Under Bing Search

Over the past few weeks, I have been noticing threads pop up in the Bing forums with complaints from webmasters that Bing's bot, aka MSNBot is not honoring their robots.txt directives.

It was not just one thread, but at least four. They include one from yesterday over here, one started on September 3rd over here, one from September 1 over here and one started on August 25th over here.

I ignored the first three, trying to give Microsoft the benefit of the doubt. Hoping it is a webmaster issue or someone spoofing MSNBot. But four threads on the same topic, all within a few weeks of each other does stand out as a possible issue.

I have personally not confirmed the issue, since I have no interest in blocking MSNBot from crawling any parts of my sites - but others don't like Bing as much as I do.

Forum discussion at Bing Forums here, here, here and over here.

 

Popular Categories

The Pulse of the search community

Search Video Recaps

 
- YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: June 20, 2025

Jun 20, 2025 - 10:00 am
Search Video Recaps

Search News Buzz Video Recap: Google Search Volatility, AI Mode in Search Console, Audio Overviews and ChatGPT Search Quality

Jun 20, 2025 - 8:01 am
Google Search Engine Optimization

Google: Larger & Longer Issues Can Take Longer To Reverse In Google Search

Jun 20, 2025 - 7:51 am
Google Search Engine Optimization

Google Troubleshooting On Blocking Content From AI Mode & AI Overviews

Jun 20, 2025 - 7:41 am
Google

Google Video Search Tests Speaker Icon With Language Label

Jun 20, 2025 - 7:31 am
Google

Google Investigating Search Clicks Leading To Translated Results

Jun 20, 2025 - 7:21 am
Previous Story: Google Webmaster Tools Keywords Report Showing Blank Lines?