Google's John Mueller said in his I/O talk that Google is now crawling "more than half of all URLs with HTTP/2." GoogleBot first started crawling over HTTP/2 for a limited number of URLs in November 2020 and now half-a-year later, over 50% of what GoogleBot crawls is done over HTTP/2.
John said this at the 4:16 mark into the video. He said "we’re excited to be able to share that we’re now crawling more than half of all URLs with HTTP/2."
How does this help you with SEO? Well, it doesn't help much out of maybe crawl budget issues, which is not an issue for the majority of the websites out there. John said "In SEO terms, this falls into the so-called crawl budget. Crawl budget is a mix of how many URLs Google wants to crawl from your website, the crawl demand, and how many your URLs our systems think your server can handle without problem, the crawl capacity. With HTTP/2 crawling, this gets a little bit more complicated because a single connection can include multiple URLs. Overall, we find that HTTP/2 crawling gives our systems the ability to request more URLs with a similar load on your servers. The decision to crawl with HTTP/2 is based on whether the server supports it and whether our systems determine there’s a possible efficiency gain. You don’t have to do anything special apart from enabling HTTPS and HTTP/2 support on your web server."
Here is the relevant slide:
John said "This means Googlebot, our crawler, won’t need to spend as much time crawling your server as before." John added "Now, since starting this experiment, we’re excited to be able to share that we’re now crawling more than half of all URLs with HTTP/2. Using stream multiplexing and header compression, the number of connections and the bandwidth have both gone down significantly. These improvements help both our crawling as well as your website serving infrastructure. Woohoo."
Here is the video:
Forum discussion at YouTube.
