(2)
Here is a little SEO bit that some of the even more experienced SEOs may not have heard of. The order of the meta robots tag is incredibly important. Did you know that the effects are cumulative when it comes to Google understanding the restrictions set in the tag?
Google's JohnMu said in a Google Webmaster Help thread:
For the robots meta tag, the effects are cumulative with regards to the restrictions, eg:<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"> <META NAME="GOOGLEBOT" CONTENT="INDEX, FOLLOW, NOODP, NOIMAGEINDEX">
would result in Googlebot treating it as a noindex, nofollow, noodp, noimageindex. This is different than the robots.txt file. You cannot provide more restrictive directives for the generic "robots" setting than for individual crawlers.
So if you need more granular control over controlling robots, use the robots.txt file.
Forum discussion at Google Webmaster Help and hat tip to Colin McDermott.
Comments:
Ben Hayes
07/30/2010 02:16 pm
nice post, makes sense, but you wouldn't think of it at the time.
Michael Martinez
07/30/2010 05:35 pm
I don't normally use a Googlebot meta tag but I always place engine-specific meta tags first before the generic robots meta tag. I'm not sure how this would help (if at all) with Googlebot. Might be worth testing to see if Google prefers the first directive over the second in such a situation since John's explanation makes it sound like that is what happens. In which case, the process is only partially cumulative.