Does this fetch my live robots.txt file?
No. Paste the content you want to test; all evaluation stays in your browser.
Développement
robots.txt Tester
Check a crawler and path against pasted robots.txt rules without uploading site configuration.
Uses specific user-agent groups and longest matching Allow or Disallow rule precedence.
✓ Les données saisies sont traitées dans ton navigateur.✓ Aucun compte
Test crawler access against robots.txt rules locally.
When to use this robots.txt tester
Use it before deploying crawl rules or while investigating a blocked path.
- Check crawler access to a path.
- Verify an Allow exception beneath a blocked folder.
- Inspect wildcard and sitemap directives privately.
Outils associés
Outils associés
FAQ
Questions fréquentes
Questions fréquentes
How are conflicting rules resolved?
The most specific matching path wins, with Allow winning when matching rules have equal length.
Does this guarantee indexing?
No. robots.txt controls crawling; indexing also depends on other page and crawler signals.