Idea To Place Google Image Ads Near Google Text Ads As Alternate | Main | Google Update & Back Link Update at Google.com

January 5, 2007

MSN Indexing Pages With No Index Commands?

There is a WebmasterWorld thread where folks are reporting that MSN Search is indexing a bit too much. Reports suggest that MSN is crawling and indexing the content found on pages which are excluded via the robots.txt directive.

Normally, if a search engine crawls the page, they may only keep the URL in their index but not the content.

Moderator Receptional adds, "My examples show no snippet. But I can get them by typing unique text into the search query."

So if they are returning results for text found on the page, maybe they are indexing a bit too much?

On the other hand, I have a thread at Search Engine Roundtable Forums suggesting MSN doesn't index enough, specifically new sites...

Forum discussion at WebmasterWorld & Search Engine Roundtable Forums.



Like The Story? Vote For It On Yahoo Buzz! Or On Sphinn!

posted rustybrick in Microsoft MSN Search at January 5, 2007 7:07 AM Comments (2)

• Google Co-op Cartoon Barry

Comments

This has happened with other search engines, the URL is there because it was found as a link on another page and put into the crawl queue.

 

Posted by Marcia at January 8, 2007 15:03

I have HUGE peoblems with MSN and their 301 redirects. Pages have been removed and cached pages just say that they have been Permanently Moved which is not true.
I used TranslationGold to translate my pages and that seems have caused me a lot of problems. Pages with query strings are not longer indexed and show up as 301'ned.... Not sure what is going on but it has happened to all my sites :(

 

Posted by George at January 12, 2007 21:10

Post a comment (Note: Can Take 120 Seconds For Your Comment To Show Up)

Do you want us to save your personal Information?

Premium Sponsors + advertise

To subscribe to the Search Engine Roundtable, click here