Google: Don't Block Your CSS & JavaScript Files

Mar 27, 2012 - 8:22 am 15 by

Yesterday, Google's Matt Cutts posted a video as a "public service announcement" asking SEOs and webmasters who are blocking GoogleBot from accessing their CSS and JavaScript files to stop blocking Google from accessing those files.

Matt Cutts said it will make for a better crawl and give Google get a better idea of what is going on with your site and return better results to the user. Here is the video:

Going forward, will you block GoogleBot from accessing your CSS and JavaScript files? Take my poll:

Here is the transcript of this video:

0:02 Hi everybody.
0:03 Matt Cutts here.
0:04 I just have a short public service announcement rather
0:07 than a question today.
0:08 And the PSA is this.
0:10 If you block Googlebot from crawling JavaScript or CSS,
0:15 please take a few minutes and just take that out of the
0:17 robots.txt.
0:18 And let us crawl the JavaScript, let us crawl the
0:20 CSS, and get a better idea of what's going on on the page.
0:24 A lot of people block it because they think, this is
0:26 going to be resources that I don't want to have the
0:29 bandwidth or something.
0:30 But Googlebot is pretty smart about not crawling stuff too
0:33 fast. And a lot of people will do things like, they'll check
0:38 for flash, but then they're including some JavaScript.
0:41 And they don't realize, including that Javascript, the
0:44 Javascript is blocked.
0:45 And so we're not able to crawl the site as effectively as we
0:48 would like.
0:49 In addition, Google is getting better at processing
0:52 JavaScript.
0:52 It's getting better at things like looking at CSS to figure
0:55 out what's important on the page.
0:58 So if you do block Googlebot, I would ask, please take a
1:01 little bit of time, go ahead and remove those blocks from
1:03 the robots.txt, so you can let Googlebot in, get a better
1:06 idea of what's going on with your site, get a better idea
1:09 of what's going on with your page.
1:10 And then that just helps everybody, in terms of if we
1:13 can find the best search results, we'll be able to
1:15 return them higher to users.
1:16 So thanks if you can take the chance.
1:18 I know it's kind of a common idiom for people to just say,
1:21 I'm going to block JavaScript and CSS.
1:23 But you don't need to do that now.
1:25 So please, in fact, actively let Googlebot crawl things
1:29 like Javascript and CSS if you can.
1:31 Thanks.

Personally, I rarely block these files from Google or anyone else.

Forum discussion at WebmasterWorld.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Google AI Overviews, Ranking Volatility, Web Filter, Google Ads AI Summaries & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: May 21, 2024

May 21, 2024 - 10:00 am
Google Search Engine Optimization

PSA: Site Reputation Abuse Policy Also Applies To Google Discover

May 21, 2024 - 7:51 am
Google

Google's Sundar Pichai On AI Overviews Killing Web, Smaller Publishers & Search Console Data

May 21, 2024 - 7:41 am
Google

Google Adds Visuals To About Source Knowledge Panels

May 21, 2024 - 7:31 am
Google

Google AI Overviews Colors Do Not Indicate Confidence Of Response

May 21, 2024 - 7:21 am
Google

Google Rolling Out More Graphical & Richer Search Results

May 21, 2024 - 7:11 am
Previous Story: Ludwig Mies van der Rohe Google Logo