Official: Bing Launches New Robots.txt Tester Tool

Sep 7, 2020 - 7:56 am 1 by
Filed Under Bing SEO

Bing Image Reporting

Back in July we reported that Bing launched, or re-launched, its robots.txt tester tool. But on Friday, September 4th, Bing posted the official announcement on its blog. I am not sure what is new since when we posted outside of the download button but maybe I am missing something. But even that looks the same to me, so I guess this is just the official announcement?

Bing wrote "Webmasters can submit a URL to the robots.txt Tester tool and it operates as Bingbot and BingAdsBot would, to check the robots.txt file and verifies if the URL has been allowed or blocked accordingly. Not only this, but the test functionality checks the URL which we have submitted against the content of the editor and hence, once changes are made in the editor, you can easily and instantly retest the URL to check for errors. The system checks for allow/disallow statements for respective user agents and displays the robots.txt file in the editor with 4 variations i.e. http://, https://, http://www, https://www. Webmaster can edit the txt file and/or download the same to be updated offline. If there have been changes to the robots file elsewhere and updated, the webmaster can use the Fetch latest option to get the latest robots file of the property."

When you click "proceed" on the tester tool:

click for full size

You get this overlay box letting you:

(1) Download the updated robots.txt file. Save the updated robots.txt file from the editor.
(2) Upload the file to your domains root. Upload the updated robots.txt to your domains root, then check your updated file is the latest version.
(3) Request Bing to update. Submit a request to let Bing know your robots.txt file has been updated.

click for full size

The download option provides a step-by-step process of updating the file that includes downloading the edited file, uploading the same on the domains root which can be checked for a live version and lastly requesting Bing to update the same.

Fabrice Canal from Microsoft said "As robots.txt rules can be extremely challenging to understand and edit especially when using wildcard character *. By offering this easy to use robots.txt testing tool, we offer transparency on how we process robots.txt and help webmasters to edit and verify their robots.txt."

Forum discussion at Twitter.


Popular Categories

The Pulse of the search community


Search Video Recaps

Google Core Update Flux, AdSense Ad Intent, California Link Tax & More - YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Google Search Engine Optimization

Google: We Need Very Few Links To Rank Pages; Links Are Less Important

Apr 22, 2024 - 7:51 am

Google Tests Discussion and Forums Questions With Answers Box

Apr 22, 2024 - 7:41 am
Google Search Engine Optimization

Forbes Blocks Its Coupon Directory Before Google Enforces The Site Reputation Abuse Policy

Apr 22, 2024 - 7:31 am
Google Maps

Google Tests Posts Button In Local Listing Business Profile

Apr 22, 2024 - 7:21 am
Google Ads

Google Search With 16 Shopping Ads

Apr 22, 2024 - 7:11 am
Search Forum Recap

Daily Search Forum Recap: April 19, 2024

Apr 19, 2024 - 4:00 pm
Previous Story: Google URL Parameter Tool Upgrade Will Be 'Really Cool' When Migrated To New Search Console