Google Clarifies 15MB Googlebot Limit - It Is A Huge Limit

Jun 30, 2022 - 7:21 am 0 by

Google Heavy

The other day, I covered how Google added a line to its Google documentation that Googlebot can crawl the first 15MB of content in an HTML file or supported text-based file, after that, it stops crawling. Then I was a bit shocked to see a large number of SEOs begin to panic.

For some reason, SEOs felt 15MB of raw HTML per page is not enough. 15MB is a massive amount of HTML on a URL by URL basis. It does not include downloading videos, images, etc, it is just the HTML source code. Again, it is a huge limit and none of this was new, it was simply just added to the documentation but has been in place at Google for a long time.

So Google's Gary Illyes did his thing to clarify and posted a nicely titled blog post on the Google blog named Googlebot and the 15 MB thing. In short, Gary explains "You, dear reader, are unlikely to be the owner of one, since the median size of a HTML file is about 500 times smaller: 30 kilobytes (kB). However, if you are the owner of an HTML page that's over 15 MB, perhaps you could at least move some inline scripts and CSS dust to external files, pretty please." He digs in more for those who are concerned, so go read it.

Then John Mueller of Google does his Twitter thread version:

Are you still concerned about this 15MB limit?

Forum discussion at Twitter.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Ongoing Google March Core Update, Googlebot To Crawl Less, Pay For Google Search AI & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: April 11, 2024

Apr 11, 2024 - 4:00 pm
Google Search Engine Optimization

Google Responds To Claims Where Search Results Can Be Harmful & Dangerous

Apr 11, 2024 - 7:51 am
Google Search Engine Optimization

Google: Differences Between 410 and 404 Is Too Minimal

Apr 11, 2024 - 7:41 am
Google Ads

Google Ads With People Also Consider Label

Apr 11, 2024 - 7:31 am
Google

Google Is Not Removing The Site Command Search Feature

Apr 11, 2024 - 7:21 am
Google Ads

Google Ads Tests Get Phone Number Button

Apr 11, 2024 - 7:11 am
Previous Story: Google Explains How To Inject Canonical Tags Using JavaScript In Updated Doc