404 Pages Should Not Return 200 Status Codes

Mar 20, 2009 • 8:09 am | comments (6) by twitter Google+ | Filed Under SEO - Search Engine Optimization
 

There are way too many sites out there that have their "Page Not Found" web pages return a 200 header status code, instead of a 404 code or a 301. The issue with showing a 200 code to spiders or others is that it means that your page that no longer exists, technically does exist. If you return a 404, it technically means the page does not exist. If you return a 301, then you say that page has been moved.

A WebmasterWorld thread has discussion around the topic of serving up a 200 status code on these not found pages. Tedster, WebmasterWorld administration, said "over time, can destroy your site's rankings. You are essentially telling Google to index the same content for an infinite number of URLs."

I'll be honest, some of our coders here set up the "Not Found" pages are 200. I then get upset and we fix it. I got really upset on Tuesday, where I made our senior developer write a script to go through all our web sites (hundreds of them) and check to see if any 404 pages returned a status code other than a 404 or 301. We found about four sites that did and we fixed all of them.

Writing a scrip that goes through your sites on a scheduled basis and checks for this type of issue might be a good idea, if you have many sites to manage. Server configurations change over time, you make changes to sites, and you add new sites. People are human and they forget to do a thing or two, here and there. So stop being human (kidding) and set up a program that checks this stuff automatically.

Why? Well, it can come back to bite you later.

Forum discussion at WebmasterWorld.

Previous story: Happy First Day of Spring :: Logos From Google & Others
 

Comments:

JohnMu

03/20/2009 12:48 pm

Thanks for bringing this topic up again, Barry! Yes, it can cause all kinds of problems... Another one that can cause similar problems is when error pages don't return an error HTTP result code (500/503). It results in search engines indexing the error messages, sometimes for quite some time. Just searching for [mysql fatal error] brings up too many pages. It's good to check what happens when your database goes down: does your site handle it properly or would you get your errors indexed instead of your content?

Michael Martinez

03/20/2009 04:18 pm

There was a time when it made sense to do this, but for several years the search engines have made it clear they are trying to identify real 404 URLs and Webmasters should be careful not to "fake" their missing content any longer. A custom 404 document should include up-to-date site navigation links or a search box to help visitors find what they are looking for.

Barry Schwartz

03/20/2009 04:23 pm

Of course Michael, 404s should have custom site nav, etc. But they should send a server status code of 404.

Ruud Hein

03/21/2009 03:19 am

ASP.NET drives me crazy in this regard. Use a custom error page and MS will *302* to the 404 and then -- get this -- because the 404 was found, the 404 will send a 200. It's as if John Cleese wrote the specs. Add a 404 status code to the 404 page -- you still have a 302 before to deal with.

Ammon Johns

03/21/2009 12:27 pm

@Ruud Ruud, the easiest fix for this (there are other alternatives) is to stop using redirection. In other words, stop trying to *send* a request for a nonexisting url to your custom 404 page, and instead bring your custom 404 page content into the page requested along with the 404 header status code.

Adrien

07/22/2009 10:54 am

Hello! I'm trying to set up a good 404.asp page in IIS 5.0. The thing is that when I selec the "File" option in the "message type " in the "custom errors" window in IIS, my 404 error message doesn't work. It seems that IIS custom error only work with .html files using the "File" option. If I use the "URL" option, it works, but then the server is returning a 200 status... So I guess that my problem is: how to assign a server status in an ASP application then?! Thanks a lot ;) ! Ad

blog comments powered by Disqus