Great news from yesterday at SES. Danny has a great roundup describing that Search Engines Unite On Sitemaps Autodiscovery at Search Engine Land and I have some more details with my coverage of the Sitemaps & URL Submission session from yesterday.
In short, all four major search engines, Google, Yahoo, Microsoft's Live.com and Ask.com will all support an autodiscovery method for Sitemaps. Sitemaps is an XML protocol that enables you to freely submit a listing of URLs with more meta-data to the search engines, so that the engines can be assisted in their crawl process. It is like a form of paid inclusion without paying.
Sitemaps was first introduced in November 2006 but back then you had to manually go to Google Webmaster Central or Yahoo Site Explorer and inform them about your sitemap. Now, all you need to do is put a little marker in your robots.txt file, telling the search engines the location of your sitemap and presto, the search engines will find it on their own.
Microsoft and Ask.com both promised to support it, but I believe are currently not supporting it yet.
More details on these bot sitemaps (not human sitemaps) at sitemaps.org.
Forum discussion at WebmasterWorld & Cre8asite Forums.
Like The Story? Vote For It On Yahoo Buzz! Or On Sphinn!
rustybrick in Search Engine Optimization at April 12, 2007 7:41 AM
Comments (2)

Comments
They finnaly got it!
Helping webmasters to get more pages indexed whithout the hassle of telling every search engine where to get it the sitemap.xml...
Now lets just hope others will follow this initiative and take up this great robots.txt protocol addition as well.
Posted by Hummerbie at April 12, 2007 11:30