Updated: Google Says Blocking Countries Outside of the US is Against Policies

Jul 2, 2008 • 7:40 am | comments (21) by twitter Google+ | Filed Under Google Search Engine Optimization
 

A Google Groups thread has a webmaster who has been receiving a lot of rogue spider attacks from the Africa region. He wants to go as far as ban the whole continent of Africa. But he is concerned that by doing so, he will also hurt his Google rankings.

It is actually not all that uncommon for network administrators to block specific regions of web traffic. In fact, I believe my office blocks the Asia and Africa regions from entering our network (not this site, but my office network). We pretty much banned that whole region, because we have no reason to allow those regions in (in most cases, but things have come up).

Would blocking the whole Africa hurt this guys search rankings in Google?

Googler, JohnMu, stepped in to say that by blocking an entire region, it would "be considered cloaking" and would be against Google's Webmaster Guidelines. Got that, if you block specific regions of traffic, like everyone outside of the US, that is cloaking and against Google's guidelines.

Do I agree with this policy? In many cases, no. If your site is local in nature and having visitors from outside a specific region doesn't make sense for your bandwidth bill, then it is up to the site owner to make that call. Of course, there may be users outside of a specific region that are your target audience, but in many cases people take the route of percentages and are willing to have some collateral damage.

John does give some excellent advice, advice that is not as easy as blocking a whole region via the Router but good advice in any event. He said, instead you should "add blocks based on the user's activity, not based on his location." Of course, then you need to build algorithms and software that detects certain activities and blocks them based on that activity. More tips on that type of detection here.

Forum discussion at Google Groups.

Update: Danny Sullivan and I talked to Google about this. Googles revised statement on this is summed up in a comment Danny, where Google said:

As long as the web server always blocks IPs from (say) Africa, it's not doing anything special/different for Googlebot, and so it wouldn't be considered cloaking, but geolocation instead.

Plus, Matt Cutts of Google gives a little more on what happened here:

Yup, what Danny said. The downside of doing a lot more talking to webmasters and site owners is that sometimes we'll misspeak, but I'd much rather have that problem and sometimes need to clarify than not be talking to webmasters as much. Barry, thanks for highlighting this, and JohnMu, thanks for always being willing to answer questions in the Google webmaster discussion group.

We have posted a new article on this retraction over here.

Previous story: Google AdSense's Optimization Report Offering Wrong Advice
 

Comments:

Michael Martinez

07/02/2008 06:26 pm

Screw Google. There is absolutely no reason to consider blocking an entire country or region as "cloaking". That's just one of the most insane and stupid things I have heard of a Googler saying.

Feydakin

07/02/2008 08:37 pm

So, I have a retail site, and I want to block anyone that I can't / won't ship to from seeing the site because of the entire customer service workload it creates, I am now a black hat?? Sweet!!

CrankyDave

07/02/2008 09:10 pm

Let's see... an ecommerce site that only ships to a very specific region, that receives fraudulent purchase attempts from a particular region they do not ship to, is going to be "cloaking" if they block that region? wow... and it's not even April 1st

Rob Abdul

07/02/2008 10:22 pm

This article has given me a great idea for a new web script. It will detect undesired activities on a site and block that user; rather then blocking an entire continent. The Google Police cannot get you for that.

Scott

07/02/2008 10:23 pm

I have seen cases where I was absolutely bombarded with SPAM form submissions all from India. The site in particular is very local in nature to my home town, and as such, I had blocked the entire country, which is my right. In doing so, my rankings were not damaged, and in fact ultimately had improved. This was a few years ago, and I have since removed the ban when I added a "captcha" to my forms - Since then I have had ZERO Spam submissions.

ignace

07/03/2008 03:11 am

This just doesn't make sense. For starters, how would Google know you are blocking a certain country if their agents that scan your site are not located in that country anyway? There is no way for me to tell -for example- if this site I am now posting on is blocked from -say- China.

JIm Jones

07/03/2008 12:47 pm

Agreed, it makes no sense at all. I dont get it.

Mr. Javo

07/03/2008 01:21 pm

This is ridiculous, Yahoo for the win

Marc Beharry

07/03/2008 01:42 pm

All the hacks i have seen have come from other countries, so I have been blocking out other countries for years for most sites. And it has been a good defense against hacking.

Thomas David Baker

07/03/2008 01:56 pm

The great virtue of the web is its openness. If you block "Africa" god knows what legitimate uses you will also block. And you'll still get attacks that you'll still have to deal with.

Marcus Breese

07/03/2008 03:29 pm

How would you expect Google to determine what you are and aren't blocking? Honestly. If you decide to block Africa, how can Google know you aren't also blocking, say, Iowa. This is cloaking. By definition. If you really don't want people from outside of your region to see your site, then add a automatic redirect (on the server, not the router) to a static page explaining this. It's not that hard to do... What if Google decides to open a data center in Capetown and decides that it will crawl the web from there? Will you block all of Africa, except the subnet that Google occupies? Really blocking access to a website based upon the region the user comes from doesn't make any sense. Now, if the user starts acting up, making 100 requests a minute, or accessing URLs that don't exist, then you might think about blocking that particular IP address, but not their entire continent.

No Name

07/03/2008 03:38 pm

Not fair. I want to block the North of England, those cheap skates never buy anything anyway.

No Name

07/03/2008 05:33 pm

Obviously, blocking ranges of IPs is not cloaking. John Mu was simply misspeaking out of his behind. End of story.

Barry Schwartz

07/03/2008 05:37 pm

Google has issued a retraction. Googlers are human and they slip up. I am glad we were able to quickly clear this up.

John S. Britsios

07/03/2008 05:42 pm

Thing have been clarified. It was a mistake: http://groups.google.com/group/Google_Webmaster_Help-Indexing/msg/aa5f095db151a067

Barry Schwartz

07/03/2008 05:45 pm

John, yes, see my update above to this post. Fresh post at http://www.seroundtable.com/archives/017603.html

Adam Moro

07/03/2008 10:57 pm

"As long as the web server always blocks IPs from (say) Africa, it's not doing anything special/different for Googlebot, and so it wouldn't be considered cloaking, but geolocation instead." You say you "talked" to Google about this? I'd sure like to know how you did that. Unless you mean that you actually spoke with Google, I can't seem to find that quote anywhere on an official google medium. Care to shed some light? And it's still cloaking. Unless webmasters have ALL of googlebot's ip addresses, how can they be sure they're showing the same thing to Google that they're showing to the users in the countries they've blocked (even if it's a 403)? Unless I've misunderstood something about geo ip databases and there are some that have a COMPLETE and ACCURATE list at ALL times, it's just not possible.

christine

07/16/2008 02:39 am

So blocking an entire geographical region is cloaking to google? Google who are now partners with the Peoples Republic of China in CENSORING the entire country of China. Well done...WTF do you call that- the invisible cloak??? What is the corporate motto - "Do No Evil" Good One. Advertising the Genocide Olympics much, I'm sure that does no evil. Sell Outs....

Paretovex

05/08/2011 02:22 pm

You need to realize that people order from U.S websites from outside of the U.S all the time to ship to U.S Addresses where the products are then picked up by aeropost service providers and they then deliver them overseas, so you don't have to worry about them "wasting" your bandwidth nearly as much as you do.

Calmarius

01/09/2013 09:53 am

Sometimes pages show up in Google's results that I cannot access. The browser says "Connection refused" or sometimes the page just won't load. But the page can be accessed through a proxy... And that's annoying. If you block an entire country except Google, your page will show up in the search results but its not accessible for the user. If Google's search results are full of inaccessible pages, that would degrade the service of the search engine, and people would use a concurrent search engine instead, that filters out these results (by installing spider bot servers in as many countries as they can). So I think blocking countries out are a good reason to give 20 page penalties for that particular site...

Henri Dufresne

03/24/2013 09:26 pm

Very good feedback Scott, i got the same issue than you. I got an online directory and received a massive number of listing spams from India (manually and automatically sent). I blocked the whole country and my moderator workload went down to ZERO.

blog comments powered by Disqus