Just over a month after launching its new Webmaster Tools, Bing has announced its enhanced robots.txt tester tool. You are now able to analyse your robots.txt file and find out if there are any issues that would stop Bing from crawling and indexing your site.
Using the new tool couldn’t be simpler. All you have to do is put the URL into the field on the tester. You can even toggle between Bingbot and Adldxbot, the crawler used by Bing Ads.
Source: bing.com
Bing’s robots.txt tester checks for allow/disallow statements, and it can check a page seconds after you’ve made any updates.
The system will also display the file in the editor with four variations:
- http://
- https://
- http://www.
- https://www.
The tester also goes even further and will guide you through the fetch and uploading process.
Bing says:
Webmasters can submit a URL to the robots.txt Tester tool and it operates as Bingbot and BingAdsBot would, to check the robots.txt file and verifies if the URL has been allowed or blocked accordingly.”
What is a robots.txt file?
When it comes to SEO, having a perfect robots.txt file is essential. It tells search engine crawlers which pages or files it can or can’t request. In fact, it’s one of the few ways you can control search engine activity on your website.
Recent SEO Posts
10 Recommended WordPress SEO Plugins
Digital marketers have often argued that WordPress is the best content management system you can really ask for, especially when
10 Proven Techniques for Independent School SEO
In today’s digital landscape, having a well-optimised website is crucial for independent schools to boost their online visibility and attract
5-Step Digital Marketing Strategy for Fundraising
As digital platforms and social networks swell with an influx of users, it’s high time for non-profit organisations to architect
Content Marketing for Charities and Fundraising
Content marketing for charities and fundraising can be a powerful tool for engaging donors, volunteers, and supporters, building trust and
Mistakes in your robots.txt file could cost you rankings – that’s one of the reasons why Bing’s robot.txt tester will be handy. It’ll spot any errors you may have missed.
Have you used Bing’s new robots.txt file tester? Let us know how you got on in the comments, or tweet us @TeamTillison.
