Reddit Blocked Bing Search & Others But Not Google

Jul 25, 2024 - 7:31 am 13 by
Filed Under Bing SEO

Reddit Bing Handcuffs

On July 1, 2024, Reddit updated its robots.txt file to prohibit most search engine and AI crawlers from accessing its content - including blocking Microsoft Bing. Reddit did not block Google Search, which we covered here a few weeks ago.

A Microsoft spokesperson told me last night, "Microsoft respects the robots.txt standard and we honor the directions provided by websites that do not want content on their pages to be used with our generative AI models. Bing stopped crawling Reddit after they implemented their updated robots.txt file on July 1, which prohibits all crawling of their site."

Reddit told The Verge, "We have been in discussions with multiple search engines. We have been unable to reach agreements with all of them, since some are unable or unwilling to make enforceable promises regarding their use of Reddit content, including their use for AI."

Mark Williams-Cook showed me that Reddit's new content was not coming up in Bing Search, he posted about this on X and then several other media outlets covered it before Microsoft sent me this statement.

Bing Reddit Index Last Week

Reddit also said, "This is not at all related to our recent partnership with Google." Yes, Google is paying Reddit $60 million annually to be able to access content on its site.

Don't forget that Google has hidden gems algorithm that helped the Reddit gain a lot of traffic from Google. Reddit is seeing a lot of bad PR around Reddit dominating Google Search. I mean, Reddit is dominating Google's search results. Heck, Google even tested special treatment for Reddit not too long ago.

In any event, Bing and many other search engine and AI bots are not permitted to crawl Reddit's site anymore. Google can and does but other engines do not.

Bold move Reddit - but with the Google licensing deal - I guess it isn't too bold of a move.

Forum discussion at X.

Update: A bit more from Jordi Ribas of Microsoft on this:

 

Popular Categories

The Pulse of the search community

Search Video Recaps

 
- YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Google

Google Search Faster By About 60 Milliseconds

Feb 13, 2025 - 7:51 am
Google Ads

Google Ads Customer Match Lists To Have 540 Days Expiration Date

Feb 13, 2025 - 7:41 am
Google Search Engine Optimization

Reddit Blames Google Algorithm Change For Weaker Than Expected Growth

Feb 13, 2025 - 7:31 am
Yahoo Search Engine

Yahoo Search AI Answers Now Showing Link Sources

Feb 13, 2025 - 7:21 am
Google

Google Search Snippet Showing Future Date: Bug Or Feature?

Feb 13, 2025 - 7:11 am
Search Forum Recap

Daily Search Forum Recap: February 12, 2025

Feb 12, 2025 - 10:00 am
Previous Story: Apple Maps Web Version Launches Beta