Google Search had some issues handling square brackets punctuation marks (i.e. [ ]) within URLs. John Mueller from Google responded to former Googler, Vanessa Fox, saying that he is "seeing some of the issues internally that came up with these kinds of characters in the URL."
He then added in round brackets saying, "(though they're 10+ years ago)." He also said, "it might be that other crawlers / indexers / fetchers aren't as exact in this logic, so it might be worth double-checking those you care about."
Vanessa Fox's original question was on Bluesky, she wrote:
I posted this question in the YT video comments: For URLs with characters that can be encoded (brackets, plus signs, spaces), does Google treat the encoded version as a distinct URL? (example.com/category?filter[]=123 vs. example.com/category?filter%5B%5D=123 used in links vs. canonical values).
John Mueller from Google had this long reply:
Well, TIL. I would have assumed they're equivalent, but they're NOT. This was a fun side-track. "?filter[]=123" ≠ "?filter%5B%5D=123" because in RFC 3986, Section 2.2 the characters brackets ([, ]) and plus signs (+) are defined as part of the reserved set."URIs that differ in the replacement of a reserved character with its corresponding percent-encoded octet are not equivalent. Percent-encoding a reserved character, or decoding a percent-encoded octet that corresponds to a reserved character, will change how the URI is interpreted by most applications. Thus, characters in the reserved set are protected from normalization..." — IETF STD 66 / RFC 3986, Section 2.2
JavaScript can even check for this: new URL('https://example.com/category?filter[]=123').href === new URL('https://example.com/category?filter%5B%5D=123').href // Output: false
That said, *maybe* your server makes them "soft-equivalent" by serving the same content. We'd crawl both versions, if we see the same content (maybe even checksum level), we'd de-dupe them and pick one.
So in short, pick one, and use it consistently (links, canonical, sitemaps, structured data). The same is with " " (space), + (plus), and "%20", but with the bonus of this being different depending on which URL element it's in (path vs query string).
I knew about the spaces part, but the brackets was unexpected, and I probably gave people the wrong answer on this in the past. Woops. BTW for hex-encoding, the case doesn't matter, so "%5B" === "%5b".
As if that wasn't enough; brackets: not only are they not equivalent, the URL with ?filter[]=123 is technically invalid (according to RFC 3986) since [ ] may only appear in the hostname to wrap raw IPv6 addresses 🤯. Browsers generally hex-encode them when you enter them, and servers are like cats.
Vanessa Fox then followed up and wrote:
Ack - I saw that! But this particular site is using the invalid form of [] throughout and it seems to be indexing fine, and it would be a ton of work to change it so they are leaving it for now... fingers crossed?
John Mueller replied:
I'd expect it to continue to work - the web is messy and not all of it is "technically correct" - but I wouldn't set up a new site like this, and I'd pay extra attention to all the internal linking (including metadata like rel=canonical, structured data, etc) to make it consistent.Also, seeing some of the issues internally that came up with these kinds of characters in the URL (though they're 10+ years ago), it might be that other crawlers / indexers / fetchers aren't as exact in this logic, so it might be worth double-checking those you care about.
So Google probably can handle it now, but it's better to be safe?
Forum discussion at Bluesky.
