New Google Search Console Robots.txt Report Replaced Robots.txt Tester Tool

Nov 16, 2023 - 7:18 am 1 by

Robot Report Servers Google Logo

Google announced a new robots.txt report within Google Search Console and at the same time said it will sunset the old robots.txt tester tool. The new tool shows which robots.txt files Google found for the top 20 hosts on your site, the last time they were crawled, and any warnings or errors encountered.

Google also said on X, "We're also making relevant info available from the Page indexing report. As part of this update, we're sunsetting the robots.txt tester."

To access this, go to the settings link for your Search Console profile or click here. This is what it looks like:

Google Search Console Robots Txt Report Setting

Here is what the report looks like for this site, yes, I have a critical error. Here are those details and note, you can request recrawl of your robots.txt file:

Google Search Console Robotstxt Report

This help document goes through a lot more about this report.

Some are not happy that the robots.txt tester tool is gone but Bing still has one:

Forum discussion at X.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Google AI Overviews, Ranking Volatility, Web Filter, Google Ads AI Summaries & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: May 17, 2024

May 17, 2024 - 4:00 pm
Search Video Recaps

Search News Buzz Video Recap: Google AI Overviews, Ranking Volatility, Web Filter, Google Ads AI Summaries & More

May 17, 2024 - 8:01 am
Google Search Engine Optimization

Remove Your Content From Google's AI Overviews

May 17, 2024 - 7:51 am
Google Ads

Google Ads AI Summaries Live For Some Advertisers

May 17, 2024 - 7:41 am
Google Maps

Order with Google For Food Delivery Going Away End Of June

May 17, 2024 - 7:31 am
Google Search Engine Optimization

Two New Googlebots: GoogleOther-Image & GoogleOther-Video

May 17, 2024 - 7:21 am
Previous Story: Bing Chat Now Named Copilot