Is Google Sending GoogleBot CSS Hunting: Google Crawling CSS Files?

Jan 5, 2007 • 7:51 am | comments (5) by | Filed Under Google Search Engine Optimization
 

A Cre8asite Forums thread links to a blog post named GoogleBot Requested a CSS File. This is not the first time I heard threads where people suspect GoogleBot is crawling their CSS files. But this one has the most discussion I have seen so far.

Some people in the thread think a manual review sparked the crawl. But I am not too sure about that.

The request in the log file looks like this:

66.249.72.52 - - [24/Oct/2006:17:17:35 -0500] “GET /global/x.css HTTP/1.1″ 200 8382 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

So it looks like Google crawled his CSS. But he never discusses if he disallowed the directory in his robots.txt file.

Anyway, for past articles on Google and CSS see:

Forum discussion at Cre8asite Forums.

Previous story: More Tidbits on Google's Duplicate Content Filter
Ninja Banner
 
blog comments powered by Disqus