Bingbot is now rolling out Brotli compression for its web crawler, Bingbot. Fabrice Canel from Microsoft said on X that Bing "enabled it on a small percentage of URLs crawled each day, and we'll gradually increase traffic throughout this spring."
As a reminder, Google supported Brotli compression for BingBot since at least 2020. This website also supports Brotli compression by the way.
Here is Fabrice's post:
Bingbot now supports Brotli compression. We’ve enabled it on a small percentage of URLs crawled each day, and we’ll gradually increase traffic throughout this spring.
— Fabrice Canel (@facan) March 12, 2024
"Brotli is a lossless data compression algorithm developed by Google. It uses a combination of the general-purpose LZ77 lossless compression algorithm, Huffman coding and 2nd-order context modelling. Brotli is primarily used by web servers and content delivery networks to compress HTTP content, making internet websites load faster," this is via Wikipedia.
Brotli was initially released on October 15, 2013, with a stable release only on October 28, 2018. Its latest stable release is version 1.1.0, published on August 31, 2023. Now almost 40% of the web uses it.
Forum discussion at X.
