Study: AI Search Engines Are Confidently Wrong Too Often

Mar 11, 2025 - 7:41 am 9 by

Magnifying Glass Search Dark

A new study from Columbia Journalism Review showed that AI search engines and chatbots, such as OpenAI's ChatGPT Search, Perplexity, Deepseek Search, Microsoft Copilot, Grok and Google's Gemini, are just wrong, way too often.

I have said this time and time again, when I see an AI Answer, at this point, I just skip over it because I know I cannot trust it and this proves that. I know it will get better over time, but until that time, I just skip reading them, because I know, well too often, it is wrong.

The study said, "Collectively, they provided incorrect answers to more than 60 percent of queries. Across different platforms, the level of inaccuracy varied, with Perplexity answering 37 percent of the queries incorrectly, while Grok 3 had a much higher error rate, answering 94 percent of the queries incorrectly."

Here is a chart showing this data by each AI search engine they tested:

Ai Search Engines Wrong Study Chart

Plus, many of these search engines and "chatbots seemed to bypass Robot Exclusion Protocol preferences."

And the study said, "Generative search tools fabricated links and cited syndicated and copied versions of articles."

Plus, they just make up citations and links, well too often:

Ai Search Engines Wrong Study Links

You can read all the data over here - but I am not at all shocked by this study, as I said above.

Forum discussion at X.

 

Popular Categories

The Pulse of the search community

Search Video Recaps

 
- YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search News

The Industry Mourns The Loss Of Jill Whalen - Content SEO Hero & Legend

Jun 22, 2025 - 1:20 pm
Search Forum Recap

Daily Search Forum Recap: June 20, 2025

Jun 20, 2025 - 10:00 am
Search Video Recaps

Search News Buzz Video Recap: Google Search Volatility, AI Mode in Search Console, Audio Overviews and ChatGPT Search Quality

Jun 20, 2025 - 8:01 am
Google Search Engine Optimization

Google: Larger & Longer Issues Can Take Longer To Reverse In Google Search

Jun 20, 2025 - 7:51 am
Google Search Engine Optimization

Google Troubleshooting On Blocking Content From AI Mode & AI Overviews

Jun 20, 2025 - 7:41 am
Google

Google Video Search Tests Speaker Icon With Language Label

Jun 20, 2025 - 7:31 am
Previous Story: Data: Google Search Had 22% Growth In Searches Year Over Year