GoogleBot Can Now Crawl From Outside The US

Nov 25, 2022 - 7:51 am 4 by

Bee Googlebot

Gary Illyes from Google announced at the Google Search Central Live in Singapore yesterday that Googlebot, Google, can now crawl from outside of the US. He said, according to reports, that the system to handle this is currently live but may not be used in all cases or it may not be used at all for your site.

Update, this report is not 100% exactly accurate. Gary posted on Mastodon after he saw this story saying "that's not quite what I said. I said that the IP blocks are allocated and you can check the countries they are assigned to, and we may use them in the future for local crawls."

Here is the rest of the original story as I previously reported it. But please take into account the update from Gary above.

Kenichi Suzuki, who was at the event, posted on Twitter saying "Googlebot can crawl from a country where the server is located (i.e. from outside US). The system has already gone live but may not be used (at all). In addition, it becomes effective only in a condition where access from outside of the country is blocked."

Here is that tweet:

We heard rumors that this might happen earlier on this year from Google's John Mueller. John said then "I could see this [where Google crawls from] expanding over time if we see a broader need." So it looks like we are there.

Keep in mind, this is probably used for sites that (a) block Googlebot access from US regions, (b) show different content from different countries and (c) Google suspects something fishy based on geo-targeting.

Google previously said that it did not find any benefit to crawling from outside of the US back in 2017. There are however times Google will crawl from a specific region but it is rare.

Forum discussion at Twitter.

Update: Valentine Pletzer posted on Mastodon that Google recently added some international IP addresses to the official list over here.

click for full size

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Google Spam Update, Volatility & Indexing Bug, AI Overviews, Google Maps Exploit, Ads, Applebot - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: June 24, 2024

Jun 24, 2024 - 10:00 am
Google Ads

Google Ads To Bring Automatically Created Assets To Ad Level?

Jun 24, 2024 - 7:51 am
Google

Google Perspectives Can Show Instagram Posts

Jun 24, 2024 - 7:41 am
Google Search Engine Optimization

Google Structured Data Carousel Guidance On How To Markup Multiple Items On Category Pages

Jun 24, 2024 - 7:31 am
Google Maps

Google Using AI Overviews On Business Profiles Local Listings

Jun 24, 2024 - 7:21 am
Google Search Engine Optimization

Google: When You Should Fix Pages 404 Status Codes

Jun 24, 2024 - 7:11 am
Previous Story: Google: 60% Of The Internet Is Duplicate