robots.txt Validator.
Parse a robots.txt, surface syntax issues, and test whether a path is allowed or blocked for a given user-agent.
Input
Output
Paste a robots.txt on the left.
About this tool.
Paste a robots.txt and the tool groups User-agent / Allow / Disallow / Crawl-delay directives, lifts Sitemap URLs to the top, and flags unknown directives, mis-ordered rules, and invalid Sitemap URLs. The path tester picks the most specific User-agent group, applies the longest-match rule with Allow winning ties, and supports the * and $ wildcards.
Privacy.
Runs in your browserParsing and path-matching run entirely in your browser. No request is made.
Disclaimer. Provided as-is, with no warranty of any kind. Always check critical results before acting on them. I accept no liability for inaccuracies, bugs, or any loss or damages arising from use of these tools.
Spotted a bug or want to suggest an improvement?Let me know
More tools.
All tools- .env ComparatorSpot missing, extra or changed keys between two env files, all inside your browser.
- CIDR / Subnet CalculatorBreak a CIDR into its network, broadcast, range, masks, and a coloured binary view.
- Semver Range TesterTest a semver range against a list of versions and see which match, plus the comparator group that matched each one.