TESTED: no need for disallow if using noindex in robots.txt. As long as @google supports it... @deepcrawl @JohnMu https://t..." />

Google: I'd Avoid Using NoIndex In Robots.txt File

Sep 9, 2015 • 8:40 am | comments (5) by twitter Google+ | Filed Under Google Search Engine Optimization

@maxxeight said you can avoid using the disallow directive if you are using a noindex within your robots.txt file.

But Google's John Mueller said, you probably shouldn't do that. He said, "I'd really avoid using the noindex" in the robots.txt file.

Here is John's response:

Robots.txt, noindex, nofollow, and all the directives out there, I found to be one of the most confused topics in the SEO space. It should be pretty straight forward but even the most seasoned SEOs have trouble with them.

Forum discussion at Twitter.

Previous story: Google Search Analytics Report Maxes Out At 999 Keywords
blog comments powered by Disqus