MSNBot acting like the old Googlebot?

May 10, 2004 • 10:13 pm | comments (0) by twitter | Filed Under Bing Search
 

Once upon a time, Google's spider ran rampant on the web, and webmasters all over took extreme measures in response.

According to a recent post at Best Practices, it seems that MSN's spider is having similar growing pains. Yikes - if I were getting hammered like that, I wouldn't rely on robots.txt, I'd block every known MSNbot IP address.

Previous story: SEM as a Business Decision
 

Comments:

No comments.

blog comments powered by Disqus