When To Use 404, NoIndex & 301 Redirects In Farmer/Panda Fix

Mar 14, 2011 • 9:03 am | comments (10) by twitter Google+ | Filed Under Google Search Engine Optimization

Google Farmer/Panda FixThe Google Farmer/Panda update hit and 40% said they were impacted by the update.

We documented advice on how to fix the issue, if you were one of the sites impacted by this update. But specifically, what do you do with the low quality content?

Do you 404 the pages so they are gone forever? Do you 301 redirect those pages to higher quality content pages? Do you noindex the pages? The answer is it depends.

Google's JohnMu made it pretty clear in a Google Webmaster Help when you should use a 404 vs a noindex vs a 301 redirect. In short, 404 the pages that you don't plan on fixing. You can noindex the pages that are currently low quality and when you improve the quality of the content on those pages, you can remove the noindex. If you have other pages you can redirect the low quality pages, then use a 301 redirect.

Here is how John put it:

  • Completely remove all pages that you absolutely don't want anymore. Let them return 404 (and make a great 404 page so that your users can get to where they were headed, or find something related). See http://www.google.com/support/webmasters/bin/answer.py?answer=93641 Yes, those pages will show up as crawl errors in Webmaster Tools, but that's fine -- they're supposed to. They won't negatively affect the rest of your site's crawling, indexing or ranking. Having pages that return 404 is fine and to be expected. Using a 410 ("Gone") HTTP result code may be a tiny bit faster, but overall you don't have to worry about the difference, a 404 is ok.
  • If you have products that you can merge, then use a 301 redirect from the alternate versions. If you can't use a 301 redirect, use the rel=canonical link element.
  • If you have entries that you want to rewrite, then using a noindex meta tag sounds like a good solution. I'd keep the URLs in the Sitemap file (you want Google to recrawl them so that the noindex meta tag is seen) and make sure that they're not disallowed by the robots.txt file. As Cristina mentioned, this means that your site will still need resources to handle the crawling of those pages, but I assume since we've crawled them in the past, that load is not a problem, right?

Once this is done, John said it still might take some time for the changes to take effect in Google. John said:

Keep in mind that any changes which you make in this regard are going to take a while to affect the algorithms. We'll have to crawl and index your updated (or removed) content, and that can take a bit of time. The better your site's URL structure is (minimal duplicate content & easy to crawl), the faster we'll be able to update things, but I certainly wouldn't expect any overnight changes. It took a while for your site to get where it's at now, and it'll take a while for those updates to be visible. On the other hand, that also gives you time to really revamp your content & site structure too :-).

There is some really solid advice in the thread.

Our ongoing coverage and stories on the Content Farmer/Panda update:

Forum discussion at Google Webmaster Help.

Previous story: Google Says Ads Don't Play Big Role In Ranking Algorithm
blog comments powered by Disqus