Google posted new developer documentation for how to authenticate requests with Web Bot Auth. This is a "new cryptographic protocol that helps websites to validate that bots are authentic," Google wrote.
Google added that this is "experimental," saying the company is "testing the protocol with some AI agents hosted on Google infrastructure."
Google defined what Web Bot Auth is, saying it is "an experimental cryptographic protocol used to authenticate requests sent by bots. Instead of relying solely on self-reported headers and IP addresses, Web Bot Auth allows agents to cryptographically sign their requests. Using Web Bot Auth helps website owners identify automated traffic on their sites, and prevents other actors from attempting to spoof reputable agents. Web Bot Auth can bring the following benefits:"
- Cryptographic certainty: Move beyond easily spoofed headers to a verified identity and decouple agent identity from IP addresses.
- Better observability: Gain clearer insights into how agents interact with your content.
- Future-proofing: Help establish a web where agent providers and websites can build mutual trust and make informed access decisions.
Google said not all Google user agents are using Web Bot Auth and Google is not yet signing every request of agents using the protocol. So keep that in mind while this is "experimental."
There is a lot more on how to do all of this in these help docs.
Forum discussion at Bluesky.
