Google: 100,000 URLs Usually Won't Impact Crawl Budget

Dec 22, 2021 - 7:11 am 1 by

Google Crawl Budget

Google's John Mueller said a site with 100,000 URLs usually won't be enough to impact crawl budget. Your site and number of pages should be larger to where you might run into an issue with Google's crawl budget.

John said this on Twitter "100k URLs is usually not enough to affect crawl budget (it's <1/minute over 3 months)." Just to be me and do what I do, Gary Illyes said in 2016 that a site with 100,000 URLs may see a benefit of using nofollow on internal links for crawl budget purposes - keep in mind, that was five years ago.

Here are the tweets in context with what John Mueller said:

Hope this thread helps some SEOs.

Forum discussion at Twitter.

 

Popular Categories

The Pulse of the search community

Search Video Recaps

 
- YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: September 12, 2025

Sep 12, 2025 - 10:00 am
Search Video Recaps

Search News Buzz Video Recap: Google Heated Spam Update, Web Declines, AI Mode Updates, AI Max Rolls Out & More

Sep 12, 2025 - 8:01 am
Google Search Engine Optimization

Google Search Quality Raters Guidelines Gains AI Overview & YMYL Definitions

Sep 12, 2025 - 7:51 am
Google

Google Search Tests Dropping 100 Search Results Parameter

Sep 12, 2025 - 7:41 am
Bing Ads

Bing Tests Microsoft Sponsored Ads With Three Dots By URL

Sep 12, 2025 - 7:31 am
Google

Google Shows AI Overviews For Currency Exchange Rates

Sep 12, 2025 - 7:21 am
 
Previous Story: Google Kwanzaa Decorations With Special Pagination Bar Now Update For Holidays