Google Might Change How NoIndex In Robots.txt Works

Apr 19, 2019 - 7:56 am 2 by


For years Google has been communicating and we've been reporting that Google does not support using the noindex direction within your robots.txt file. Well, people still use it and now Gary Illyes from Google is on the case - he may end up making sure it completely doesn't work.

In short, John was asked about it again and he gave the same answer he has been giving for years:

So then Gary Illyes stepped in and said this may go away soon as he reviews the code behind it:

Why fully remove support for this? Well, (1) Google has been telling people not to use it and (2) Gary said "Technically, robots.txt is for crawling. The meta tags are for indexing. During indexing they'd be applied at the same stage so there's no good reason to have both of them."

Then people said but people use it, don't drop it. So he said he will investigate and if that is true, he will try to make a pitch to keep it:

He goes on:

I bet the unofficial support does go away at some point - so just don't use it!

Forum discussion at Twitter.


Popular Categories

The Pulse of the search community


Search Video Recaps

Google Core Update Flux, AdSense Ad Intent, California Link Tax & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: April 19, 2024

Apr 19, 2024 - 4:00 pm
Search Video Recaps

Search News Buzz Video Recap: Google Core Update Flux, AdSense Ad Intent, California Link Tax & More

Apr 19, 2024 - 8:01 am
Google Ads

Google Tests More Google Ad Card Formats

Apr 19, 2024 - 7:51 am
Google Search Engine Optimization

Google: It's Unlikely Your Rankings Dropped Because You Have Two Websites

Apr 19, 2024 - 7:41 am
Google Search Engine Optimization

Google: Indexing API May Work For Unsupported Content But...

Apr 19, 2024 - 7:31 am
Google Search Engine Optimization

Google: Are Hyphenated Domains Bad For Google Rankings?

Apr 19, 2024 - 7:21 am
Previous Story: Google Posts Supports Highlighting Customer Testimonials & Reviews