Is Google Sending GoogleBot CSS Hunting: Google Crawling CSS Files?

Jan 5, 2007 - 7:51 am 5 by

A Cre8asite Forums thread links to a blog post named GoogleBot Requested a CSS File. This is not the first time I heard threads where people suspect GoogleBot is crawling their CSS files. But this one has the most discussion I have seen so far.

Some people in the thread think a manual review sparked the crawl. But I am not too sure about that.

The request in the log file looks like this:

66.249.72.52 - - [24/Oct/2006:17:17:35 -0500] “GET /global/x.css HTTP/1.1″ 200 8382 “-” “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

So it looks like Google crawled his CSS. But he never discusses if he disallowed the directory in his robots.txt file.

Anyway, for past articles on Google and CSS see:

Forum discussion at Cre8asite Forums.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
- YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Google

Google Search Having Indexing Issues Again

Dec 9, 2024 - 7:13 pm
Search Forum Recap

Daily Search Forum Recap: December 9, 2024

Dec 9, 2024 - 10:00 am
Google Updates

Google Search Ranking Volatility Continues Post Core Update

Dec 9, 2024 - 8:01 am
Google Search Engine Optimization

Google Adds FAQs For Site Reputation Abuse Policy Including Moving Content

Dec 9, 2024 - 7:51 am
Google Search Engine Optimization

Google Search Console Links Report With Large Change In Links?

Dec 9, 2024 - 7:41 am
Google

Watch Google CEO Sundar Pichai Interview Say Search To Profoundly Change In 2025

Dec 9, 2024 - 7:31 am
Previous Story: More Tidbits on Google's Duplicate Content Filter