Official: Bing Launches New Robots.txt Tester Tool

Sep 7, 2020 - 7:56 am 1 by
Filed Under Bing SEO

Bing Image Reporting

Back in July we reported that Bing launched, or re-launched, its robots.txt tester tool. But on Friday, September 4th, Bing posted the official announcement on its blog. I am not sure what is new since when we posted outside of the download button but maybe I am missing something. But even that looks the same to me, so I guess this is just the official announcement?

Bing wrote "Webmasters can submit a URL to the robots.txt Tester tool and it operates as Bingbot and BingAdsBot would, to check the robots.txt file and verifies if the URL has been allowed or blocked accordingly. Not only this, but the test functionality checks the URL which we have submitted against the content of the editor and hence, once changes are made in the editor, you can easily and instantly retest the URL to check for errors. The system checks for allow/disallow statements for respective user agents and displays the robots.txt file in the editor with 4 variations i.e. http://, https://, http://www, https://www. Webmaster can edit the txt file and/or download the same to be updated offline. If there have been changes to the robots file elsewhere and updated, the webmaster can use the Fetch latest option to get the latest robots file of the property."

When you click "proceed" on the tester tool:

click for full size

You get this overlay box letting you:

(1) Download the updated robots.txt file. Save the updated robots.txt file from the editor.
(2) Upload the file to your domains root. Upload the updated robots.txt to your domains root, then check your updated file is the latest version.
(3) Request Bing to update. Submit a request to let Bing know your robots.txt file has been updated.

click for full size

The download option provides a step-by-step process of updating the file that includes downloading the edited file, uploading the same on the domains root which can be checked for a live version and lastly requesting Bing to update the same.

Fabrice Canal from Microsoft said "As robots.txt rules can be extremely challenging to understand and edit especially when using wildcard character *. By offering this easy to use robots.txt testing tool, we offer transparency on how we process robots.txt and help webmasters to edit and verify their robots.txt."

Forum discussion at Twitter.

 

Popular Categories

The Pulse of the search community

Search Video Recaps

 
- YouTube
Video Details More Videos Subscribe to Videos

Most Recent Articles

Google Ads

Google Ads Come To AI Mode & AI Overviews Desktop

May 22, 2025 - 7:51 am
Google Ads

New Google Ads Smart Bidding Exploration: Bidding Update

May 22, 2025 - 7:41 am
Google Ads

Google Ads New AI Tools & Agentic Capabilities

May 22, 2025 - 7:31 am
Google Search Engine Optimization

New Google Docs On Using AI Generative Content & AI Search Features

May 22, 2025 - 7:21 am
Bing Search

Bing Tests Double Border Search Box

May 22, 2025 - 7:11 am
Search Forum Recap

Daily Search Forum Recap: May 21, 2025

May 21, 2025 - 10:00 am
Previous Story: Google URL Parameter Tool Upgrade Will Be 'Really Cool' When Migrated To New Search Console