GoogleBot Can Now Crawl From Outside The US

Nov 25, 2022 - 7:51 am 4 by

Bee Googlebot

Gary Illyes from Google announced at the Google Search Central Live in Singapore yesterday that Googlebot, Google, can now crawl from outside of the US. He said, according to reports, that the system to handle this is currently live but may not be used in all cases or it may not be used at all for your site.

Update, this report is not 100% exactly accurate. Gary posted on Mastodon after he saw this story saying "that's not quite what I said. I said that the IP blocks are allocated and you can check the countries they are assigned to, and we may use them in the future for local crawls."

Here is the rest of the original story as I previously reported it. But please take into account the update from Gary above.

Kenichi Suzuki, who was at the event, posted on Twitter saying "Googlebot can crawl from a country where the server is located (i.e. from outside US). The system has already gone live but may not be used (at all). In addition, it becomes effective only in a condition where access from outside of the country is blocked."

Here is that tweet:

We heard rumors that this might happen earlier on this year from Google's John Mueller. John said then "I could see this [where Google crawls from] expanding over time if we see a broader need." So it looks like we are there.

Keep in mind, this is probably used for sites that (a) block Googlebot access from US regions, (b) show different content from different countries and (c) Google suspects something fishy based on geo-targeting.

Google previously said that it did not find any benefit to crawling from outside of the US back in 2017. There are however times Google will crawl from a specific region but it is rare.

Forum discussion at Twitter.

Update: Valentine Pletzer posted on Mastodon that Google recently added some international IP addresses to the official list over here.

click for full size

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: May 24, 2024

May 24, 2024 - 10:00 am
Search Video Recaps

Search News Buzz Video Recap: Google Ranking Volatility, Ads In Google AI Overviews, Sundar Pichai Interview, Heartfelt Helpful Content & More Ad News

May 24, 2024 - 8:01 am
Google Search Engine Optimization

Google: The Site Reputation Abuse Policy Enforcement Not Yet Algorithmic

May 24, 2024 - 7:51 am
Google Search Engine Optimization

Google Search Can Now Index Electronic Publication (EPUB)

May 24, 2024 - 7:41 am
Google

Directory Of Embarrassing Google AI Overviews

May 24, 2024 - 7:31 am
Web Analytics

Google Analytics Real-Time Reports Adds Users In Last 5 Minutes

May 24, 2024 - 7:21 am
Previous Story: Google: 60% Of The Internet Is Duplicate