Google: Firewalls or CDNs Issues Are The Most Common Reason For Blocking Googlebot

Feb 1, 2023 - 7:51 am 2 by

Google Fire Wall

Gary Illyes from Google posted a new PSA on LinkedIn saying that the most common reason a site unexpectedly blocks Googlebot from crawling is due to a misconfiguration of a firewall or CDN.

Gary wrote, "check what traffic your firewalls and CDN are blocking." "By far the most common issue in my inbox is related to firewalls or CDNs blocking googlebot traffic. If I reach out to the blocking site, in the vast majority of the cases the blockage is unintended."

So what can you do? Gary said, "I've said this before, but want to emphasize it again: make a habit of checking your block rules. We publish our IP ranges so it should be very easy to run an automation that checks the block rules against the googlebot subnets."

Gary linked to this help document for more details.

In short, do what you can to test to see if your site is accessible to Googlebot. You can use the URL inspection tool in Google Search Console, as one method. Also, confirm with your CDN or firewall company that they are allowing Googlebot and ask them to prove it.

Forum discussion at on LinkedIn.

 

Popular Categories

The Pulse of the search community

Search Video Recaps

 
- YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Google

Google Search Bitcoin Charts Gone - It's A Bug

Feb 9, 2025 - 11:45 am
Search Forum Recap

Daily Search Forum Recap: February 7, 2025

Feb 7, 2025 - 10:00 am
Search Video Recaps

Search News Buzz Video Recap: Google Volatility, Google Ads Quality Update, Google AI Overviews Updates & Other Search Engines

Feb 7, 2025 - 8:01 am
Google Search Engine Optimization

Google Says People Know When Author Bios Are For SEO Purposes

Feb 7, 2025 - 7:51 am
Google Ads

Google Ads Keyword Planner New Auto-Organize Keywords To Ad Groups

Feb 7, 2025 - 7:41 am
Google Maps

Google Business Profiles Reinstatements Now Include Case & 5 Upload Documents

Feb 7, 2025 - 7:31 am
Previous Story: Google: Search Console Verification Does Not Impact Your Ranking