Reddit Blocked Bing Search & Others But Not Google

Jul 25, 2024 - 7:31 am 13 by
Filed Under Bing SEO

Reddit Bing Handcuffs

On July 1, 2024, Reddit updated its robots.txt file to prohibit most search engine and AI crawlers from accessing its content - including blocking Microsoft Bing. Reddit did not block Google Search, which we covered here a few weeks ago.

A Microsoft spokesperson told me last night, "Microsoft respects the robots.txt standard and we honor the directions provided by websites that do not want content on their pages to be used with our generative AI models. Bing stopped crawling Reddit after they implemented their updated robots.txt file on July 1, which prohibits all crawling of their site."

Reddit told The Verge, "We have been in discussions with multiple search engines. We have been unable to reach agreements with all of them, since some are unable or unwilling to make enforceable promises regarding their use of Reddit content, including their use for AI."

Mark Williams-Cook showed me that Reddit's new content was not coming up in Bing Search, he posted about this on X and then several other media outlets covered it before Microsoft sent me this statement.

Bing Reddit Index Last Week

Reddit also said, "This is not at all related to our recent partnership with Google." Yes, Google is paying Reddit $60 million annually to be able to access content on its site.

Don't forget that Google has hidden gems algorithm that helped the Reddit gain a lot of traffic from Google. Reddit is seeing a lot of bad PR around Reddit dominating Google Search. I mean, Reddit is dominating Google's search results. Heck, Google even tested special treatment for Reddit not too long ago.

In any event, Bing and many other search engine and AI bots are not permitted to crawl Reddit's site anymore. Google can and does but other engines do not.

Bold move Reddit - but with the Google licensing deal - I guess it isn't too bold of a move.

Forum discussion at X.

Update: A bit more from Jordi Ribas of Microsoft on this:

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
- YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Google Search Engine Optimization

Google Search Snippet With Reviewed By Author Name

Nov 7, 2024 - 7:31 am
Google Search Engine Optimization

Google On When To Split Load Across Subdomains Or Other Domains

Nov 7, 2024 - 7:21 am
Google Ads

Google Ads Continues To Test Ads Mixed Within Organic Results - Dynamic Ad Placement

Nov 7, 2024 - 7:11 am
Search Forum Recap

Daily Search Forum Recap: November 6, 2024

Nov 6, 2024 - 10:00 am
Google Updates

Election 2024 Google Search Ranking Update 🇺🇸

Nov 6, 2024 - 7:51 am
Google Search Engine Optimization

Report: 22% Of Sites Hit By Google's Helpful Content Update Are Up 20%+

Nov 6, 2024 - 7:41 am
Previous Story: Apple Maps Web Version Launches Beta