GoogleBot Can Now Crawl From Outside The US

Nov 25, 2022 - 7:51 am 4 by

Bee Googlebot

Gary Illyes from Google announced at the Google Search Central Live in Singapore yesterday that Googlebot, Google, can now crawl from outside of the US. He said, according to reports, that the system to handle this is currently live but may not be used in all cases or it may not be used at all for your site.

Update, this report is not 100% exactly accurate. Gary posted on Mastodon after he saw this story saying "that's not quite what I said. I said that the IP blocks are allocated and you can check the countries they are assigned to, and we may use them in the future for local crawls."

Here is the rest of the original story as I previously reported it. But please take into account the update from Gary above.

Kenichi Suzuki, who was at the event, posted on Twitter saying "Googlebot can crawl from a country where the server is located (i.e. from outside US). The system has already gone live but may not be used (at all). In addition, it becomes effective only in a condition where access from outside of the country is blocked."

Here is that tweet:

We heard rumors that this might happen earlier on this year from Google's John Mueller. John said then "I could see this [where Google crawls from] expanding over time if we see a broader need." So it looks like we are there.

Keep in mind, this is probably used for sites that (a) block Googlebot access from US regions, (b) show different content from different countries and (c) Google suspects something fishy based on geo-targeting.

Google previously said that it did not find any benefit to crawling from outside of the US back in 2017. There are however times Google will crawl from a specific region but it is rare.

Forum discussion at Twitter.

Update: Valentine Pletzer posted on Mastodon that Google recently added some international IP addresses to the official list over here.

click for full size

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Google Core Update Flux, AdSense Ad Intent, California Link Tax & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Blog Administration

Programming Note: Offline For Passover

Apr 22, 2024 - 4:05 pm
Search Forum Recap

Daily Search Forum Recap: April 22, 2024

Apr 22, 2024 - 4:00 pm
Google Search Engine Optimization

Google: We Need Very Few Links To Rank Pages; Links Are Less Important

Apr 22, 2024 - 7:51 am
Google

Google Tests Discussion and Forums Questions With Answers Box

Apr 22, 2024 - 7:41 am
Google Search Engine Optimization

Forbes Blocks Its Coupon Directory Before Google Enforces The Site Reputation Abuse Policy

Apr 22, 2024 - 7:31 am
Google Maps

Google Tests Posts Button In Local Listing Business Profile

Apr 22, 2024 - 7:21 am
Previous Story: Google: 60% Of The Internet Is Duplicate