Google: Nofollow Can Be Used For Crawl Prioritization

Jun 21, 2012 • 8:27 am | comments (11) by twitter Google+ | Filed Under Google Search Engine Optimization
 

GooglebotBack in 2009 there was a huge spike in concern over how Google handled nofollowed links where the PageRank would be diluted anyway when used. Matt Cutts explained then that if you had 5 links and 1 of the 5 were nofollowed, still the PageRank would be divided up as 5 and not 4.

I spotted a recent Google Webmaster Help thread where Googler John Mueller said the nofollow attribute can be used for crawl prioritization purposes.

John said it can be used "to some extent, for crawl prioritization." Now, the words "to some extent" is where John is referencing the PageRank flow issue noted in 2009.

But yes, you would be able to use the nofollow for helping GoogleBot crawl your site more efficiently. Honestly, I doubt Google or John would recommend this in most cases, as opposed to deploying a more efficient navigation scheme - but here is Google and John saying it can be used for this purpose as well.

Crawl prioritization is basically a way to help GoogleBot crawl your site in the most efficient manner. This helps with GoogleBot discovering new or updated content, plus PageRank and link juice distribution. Typically for larger sites this is a key area of SEO focus.

Forum discussion at Google Webmaster Help.

Previous story: Export Google Webmaster Tools Data To Google Spreadsheets
 

Comments:

Yousaf

06/21/2012 12:31 pm

Funny, Matt Cutts has always been against this.

Webstats Art

06/21/2012 02:46 pm

Why has Matt Cutts been against crawl prioritization through nodofollow links. Can someone expand on this?

Yw

06/21/2012 03:13 pm

Well, I might falls on the camp of thought that even if it might have impact on crawl priorisation the impact will be so insignificant that our resources can be better spent on some other areas instead of optimising the nofollow links..

Michael Martinez

06/21/2012 03:53 pm

Matt is against it because no SEO is able to implement it correctly.

Michael Martinez

06/21/2012 03:53 pm

Here we go with PageRank Sculpting again. People are so bound and determined to screw up their Websites -- they really need to leave "crawl prioritization" in the dustbin of really bad SEO ideas.

Coryon Redd

06/21/2012 08:35 pm

I run an ecommerce site that sells batteries with over 100,000k pages. We customize each page to some extent but many are for products that fit multiple devices and therefore have many duplicate pages. For example, one cell phone battery could have 100 product detail pages with similar but not exactly the same content. We built this design for both users and SEO but have had difficulty over our 13 years in business training google to pick up the most important pages without penalizing our overall SEO strength. PageRank sculpting made sense for us (when it worked), because we are smart enough to limit google to all but the top few pages. I sure hope "crawl prioritization" allows us to make better use of our wasted linkjuice.

Alan

06/22/2012 08:37 am

This is true. Matt even goes as far as to say in one video just leave all internal links follow , playing with nofollow will do more harm than good. I can see how this could be true as just because a page you want to nofollow has low pr does not mean google think it is not an authoritative page on the site. People are so hung up on PR it is scary.

Farfel

06/22/2012 02:03 pm

Ha! I knew as soon as I started reading this article that after the jump I would see the profile pic of Michael Martinez hand poised under chin! Go get 'em MM!!!

Yousaf

06/22/2012 02:22 pm

Alan, you hit the nail on its head. I would like to add a couple of more points. 1. Concentrate on your site structure/architecture. If you improve that you don't need to do PageRank sculpting (nofollowing internal links). 2. If you can't be bothered to improve your IA/site structure then focus on positioning internal links above the fold on important pages. BTW Here is Matt Cutts's video http://www.youtube.com/watch?v=R4IE4WLPLZQ

John Britsios

06/22/2012 06:54 pm

Does you mean it is possible to be implemented but no SEO knows how? I brought up this piece at the end of discussion I started two weeks ago https://plus.google.com/115194199565322841506/posts/eMrTRk3ZDoG. Would you mind having a look and share your thoughts? If you do, I must clarify in advance that we were always on the same page about the topic PageRank sculpting.

Bill Sebald

06/22/2012 07:44 pm

I've used nofollows to keep Google from spidering supplemental pages where I couldn't meta robots out due to an enterprise CMS limitation, and to support the robots.txt that won't necessary keep something out the index. I was having this problem of too much search traffic on these pages with 100% bounce, so on the new sites built with the same CMS I added a couple specific nofollow, and the problem went away. I didn't screw it up. It forced Google to serve a more relevant page for 85% of the queries that were dying on this page. That's what Mueller is talking about.

blog comments powered by Disqus