Google's New Search Snippets Blocked By Robots.txt Warning

Aug 16, 2012 - 8:23 am 35 — by

A description for this result is not available because of this site's robots.txt

Google is showing a more descriptive search snippet for when it cannot show any search snippet in the search results.

If a page is being blocked by the robots.txt file, Google won't show the search snippet (description) in the search results. Google may show the page in the search results because of the linkage data it has. The example in the picture above is the only URL I have blocked from Google indexing it. I regret writing it, but also would regret removing it, so I blocked it from Google.

If you try to match on it within Google, it turns the notification:

A description for this result is not available because of this site's robots.txt – learn more.

The learn more link goes to the page named Block or remove pages using a robots.txt file in Google's help section.

I think this provides a much better search experience for someone who sees these blocked pages.

I do wonder if they show up for non site: command searches, I'd assume so.

Forum discussion at WebmasterWorld.

 

Popular Categories

The Pulse of the search community

Google Search Volatility

More Details

Search Video Recaps

 
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: September 25, 2026

Sep 25, 2026 - 10:00 am
Search Video Recaps

Search News Buzz Video Recap: Google September 2026 Spam Update Is Here, Goodbye Discover Traffic, Multimodal Search Reporting & CrUX Ad Metrics In Page Experience

Sep 25, 2026 - 8:01 am
OpenAI

OpenAI Has Quality Raters, AI Trainers, & They Cheated With AI

Sep 25, 2026 - 7:51 am
Google Ads

Google LSAs Showing Google Business Profile Number, Not Google Ads Number

Sep 25, 2026 - 7:41 am
Bing Search

Bing Testing Editable Labels On Images

Sep 25, 2026 - 7:31 am
Google Ads

Google Ads Disables Editing Target ROAS While Promotion Mode Is Active

Sep 25, 2026 - 7:21 am
Previous Story: Email Delivery Bug With Google Webmaster Tools Messages