Google launched shared conversations in Google Bard about a week ago and forgot to ensure search engines can't index the results. The robots.txt had(s) nothing in it to block bard.google.com/share from being indexed and since Google Search likes to index things, it indexed those public conversations.
Here is a screenshot of what I saw yesterday afternoon (click to enlarge):
Note, this morning when the story went live, Google is showing twice as many URLs in its index than it did yesterday.
This was spotted first by Gagan Ghotra and he posted about it on X. I asked Danny Sullivan, the Google Search Liason about this, saying Google Search can't really be interested in indexing this and he said I am right.
Sullivan wrote, "Bard allows people to share chats, if they choose. We also don't intend for these shared chats to be indexed by Google Search. We're working on blocking them from being indexed now."
Here are those tweets:
Bard allows people to share chats, if they choose. We also don't intend for these shared chats to be indexed by Google Search. We're working on blocking them from being indexed now.
— Google SearchLiaison (@searchliaison) September 26, 2023
I suspect Bard will add this to its robots.txt file to block it from being crawled by Google Search? Or maybe Google will do something different to block these chats from showing up in Google Search?
The wild thing is that if Google indexes these shared conversations, it can alter what Google Search validates as being accurate in Bard - remember this?
And yea, some of those Bard conversations were selected as featured snippets by Google Search:
Google Bard conversation ranking as snippet 😄
— Gagan Ghotra (@gaganghotra_) September 26, 2023
Query - why google is not indexing my blogger posts fast. https://t.co/LlsejAdVeV pic.twitter.com/B9Kxp9kfh8
But John won't be helping:
Given our honest results policy, the folks from Search are also not going to help them to figure this out. There's public documentation on how verification, noindex, robots.txt, etc works.
— John, aka "a total bell cheese" (@JohnMu) September 26, 2023
This kinda goes into it: https://t.co/ol6JSb7GHq -- though tbh it's been a while since I listened in.
— John, aka "a total bell cheese" (@JohnMu) September 26, 2023
Forum discussion at X.
Update: Later today, the Bard team blocked these conversations using robots.txt:
It's blocked now via robots.txt. Note, urls can still be indexed when blocked by robots.txt, just w/out Google crawling the content. But Google shouldn't surface those often in the SERPs (but it can...) Just a heads up. Also, they should use the removals tool for what's indexed: https://t.co/oQM84YvrfU pic.twitter.com/N2kiTMF5Zd
— Glenn Gabe (@glenngabe) September 27, 2023
Update 2: The Bard results are gone from Google Search:
Bard shared conversations no longer showing up in Google Search for me https://t.co/eRKfGtEC7O - original story at https://t.co/GmWAoXMMYM pic.twitter.com/Gu6wLlBphq
— Barry Schwartz (@rustybrick) September 28, 2023
Update 3: Jack from the Bard team chimed in, he said:
We know that people put their trust in Google, and we take that responsibility very seriously. That’s why, by default, Bard chats are not publicly accessible.We do allow users to publicly share their own Bard chats, if they choose; a given chat is publicly shared only after the user confirms that they want to create a public link with that chat.
Of all the chats that users have chosen to publicly share, only a subset were indexed in Google. These indexed chats were those that users had chosen to publicly share and that were also linked to from blog posts, forums or other public web pages.
We’ve now removed those chats from Google Search, and blocked users’ shared chats from being listed in search results.
We know that people put their trust in Google, and we take that responsibility very seriously. That’s why, by default, Bard chats are not publicly accessible.
— Jack Krawczyk (@JackK) September 28, 2023
We do allow users to publicly share their own Bard chats, if they choose; a given chat is publicly shared only after…
Then Glenn Gabe responded, to which Danny Sullivan from Google responded:
While I’m not on the Bard team, I know that removing Google URLs from Google can sometimes be more complicated than for non-Google sites. You’re going to see some short term changes in the blocking that should settle down into what makes sense for non-Google sites.
— Google SearchLiaison (@searchliaison) September 28, 2023
