Google Working On Improving Robots.txt Tester Tool

Jan 7, 2021 - 7:51 am 0 by

Google Search Central

Gary Illyes from Google confirmed he is slowly working on improving the robots.txt tester tool, which is currently available on the old version of Google Search Console. He said on Twitter "we're currently in the process(*) of integrating the tester with the production version of the parser."

Gary said this because there was a complaint that there is a bug with either the tester tool or how Google reads a site's robots.txt file. Gary said it might be just the tool itself and when the tool is updated to use the production version of the parser, you will know for sure.

A reminder that Gary was involved in the robots exclusion protocol standard project and then open sourcing the parser code.

Here is Gary's tweet:

If you scroll up through the Twitter thread, you can learn more about the complaint.

In 2014, Google revamped its robots.txt tester tool but it has been updated numerous times before and after that time.

Forum discussion at Twitter.

 

Popular Categories

The Pulse of the search community

Follow

Search Video Recaps

 
Google Core Update Rumbling, Manual Actions FAQs, Core Web Vitals Updates, AI, Bing, Ads & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Search Forum Recap

Daily Search Forum Recap: March 15, 2024

Mar 15, 2024 - 4:00 pm
Search Video Recaps

Search News Buzz Video Recap: Google Core Update Rumbling, Manual Actions FAQs, Core Web Vitals Updates, AI, Bing, Ads & More

Mar 15, 2024 - 8:01 am
Google Updates

Google March 2024 Core & Spam Update Movement Today

Mar 15, 2024 - 7:51 am
Google

Google Image Search Results Testing Like Button

Mar 15, 2024 - 7:41 am
Google

Google Merchant Center Product Studio With Themed Templates Including St. Patrick's Day

Mar 15, 2024 - 7:31 am
Google Maps

Google Search Dishes Nearby Carousel

Mar 15, 2024 - 7:21 am
Previous Story: Google Data Studio Can Pull Search Console News Tab Queries