Google: Indexing Search Result Pages Is A Watering Down Indexed Pages Issue

Mar 10, 2020 - 7:42 am 2 by

Google Water Down

Google's John Mueller responded to a comment about Google's guidelines around "Use the robots.txt file on your web server to manage your crawling budget by preventing crawling of infinite spaces such as search result pages." He said this is less about spam and more about " watering down your indexed content with useless pages that compete with each other."

He posted this on Twitter:

Here is the original tweet from Lily Ray from this conversarion:

In 2007, Google told webmasters to block internal search results from being indexed. The original guideline read "Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don’t add much value for users coming from search engines." Now it reads "Use the robots.txt file on your web server to manage your crawling budget by preventing crawling of infinite spaces such as search result pages."

Then ten years later, Google's John Mueller explained why Google doesn't want your search result pages in its index. He said "they make infinite spaces (crawling), they're often low-quality pages, often lead to empty search results/soft-404s."

So it really wasn't always about spam but blocking pages that might not be as relevant to Google.

Forum discussion at Twitter.

 

Popular Categories

The Pulse of the search community

Search Video Recaps

 
- YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: July 11, 2025

Jul 11, 2025 - 10:00 am
Search Video Recaps

Search News Buzz Video Recap: Google Helpful Content Recoveries With Core Update, AI Mode Expands, Hidden Ad Terms & Fewer Ad Clicks

Jul 11, 2025 - 8:01 am
Google Maps

Google Local & Business Profiles Drops June 25th - Reporting Glitch?

Jul 11, 2025 - 7:51 am
Google

Google Tests Zoom Effect In On Images Within Search Results

Jul 11, 2025 - 7:41 am
Bing Search

Bing Search Image Box Tests Explore Image & Visual Search Buttons

Jul 11, 2025 - 7:31 am
Bing Search

Microsoft Bing Tests Delivery Truck Icon In Search Ads

Jul 11, 2025 - 7:21 am
Previous Story: Google Search Relations Team Grounded During Coronavirus But Will Help Remotely