Jeff Atwood, co-founder of Stack Overflow, posted on Twitter the other day that he once checked in a robots.txt that deindexed all Stack Overflow content from Google. In which, Matt Cutts, the ex-Googler, responded that "to be fair, a lot of people do this," in order to cheer him up a bit.
Honestly, it is very common - I see it all the time in the Google support forums. I won't say that I have ever done that but I will say it has been done many many times by savvy developers.
@codinghorror to be fair, a lot of people do this.
— Matt Cutts (@mattcutts) March 3, 2017
So when a site by a seasoned set of developers as well known as Stack Overflow makes the mistake, it makes others in the business feel a bit better.
Forum discussion at Twitter.
