Google: Don't Block Your CSS & JavaScript Files

Mar 27, 2012 - 8:22 am 15 by

Yesterday, Google's Matt Cutts posted a video as a "public service announcement" asking SEOs and webmasters who are blocking GoogleBot from accessing their CSS and JavaScript files to stop blocking Google from accessing those files.

Matt Cutts said it will make for a better crawl and give Google get a better idea of what is going on with your site and return better results to the user. Here is the video:

Going forward, will you block GoogleBot from accessing your CSS and JavaScript files? Take my poll:

Here is the transcript of this video:

0:02 Hi everybody.
0:03 Matt Cutts here.
0:04 I just have a short public service announcement rather
0:07 than a question today.
0:08 And the PSA is this.
0:10 If you block Googlebot from crawling JavaScript or CSS,
0:15 please take a few minutes and just take that out of the
0:17 robots.txt.
0:18 And let us crawl the JavaScript, let us crawl the
0:20 CSS, and get a better idea of what's going on on the page.
0:24 A lot of people block it because they think, this is
0:26 going to be resources that I don't want to have the
0:29 bandwidth or something.
0:30 But Googlebot is pretty smart about not crawling stuff too
0:33 fast. And a lot of people will do things like, they'll check
0:38 for flash, but then they're including some JavaScript.
0:41 And they don't realize, including that Javascript, the
0:44 Javascript is blocked.
0:45 And so we're not able to crawl the site as effectively as we
0:48 would like.
0:49 In addition, Google is getting better at processing
0:52 JavaScript.
0:52 It's getting better at things like looking at CSS to figure
0:55 out what's important on the page.
0:58 So if you do block Googlebot, I would ask, please take a
1:01 little bit of time, go ahead and remove those blocks from
1:03 the robots.txt, so you can let Googlebot in, get a better
1:06 idea of what's going on with your site, get a better idea
1:09 of what's going on with your page.
1:10 And then that just helps everybody, in terms of if we
1:13 can find the best search results, we'll be able to
1:15 return them higher to users.
1:16 So thanks if you can take the chance.
1:18 I know it's kind of a common idiom for people to just say,
1:21 I'm going to block JavaScript and CSS.
1:23 But you don't need to do that now.
1:25 So please, in fact, actively let Googlebot crawl things
1:29 like Javascript and CSS if you can.
1:31 Thanks.

Personally, I rarely block these files from Google or anyone else.

Forum discussion at WebmasterWorld.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Google Core Update Rumbling, Manual Actions FAQs, Core Web Vitals Updates, AI, Bing, Ads & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: March 15, 2024

Mar 15, 2024 - 4:00 pm
Search Video Recaps

Search News Buzz Video Recap: Google Core Update Rumbling, Manual Actions FAQs, Core Web Vitals Updates, AI, Bing, Ads & More

Mar 15, 2024 - 8:01 am
Google Updates

Google March 2024 Core & Spam Update Movement Today

Mar 15, 2024 - 7:51 am
Google

Google Image Search Results Testing Like Button

Mar 15, 2024 - 7:41 am
Google

Google Merchant Center Product Studio With Themed Templates Including St. Patrick's Day

Mar 15, 2024 - 7:31 am
Google Maps

Google Search Dishes Nearby Carousel

Mar 15, 2024 - 7:21 am
Previous Story: Ludwig Mies van der Rohe Google Logo