Does this fetch my live robots.txt file?
No. Paste the content you want to test; all evaluation stays in your browser.
Programowanie
robots.txt Tester
Check a crawler and path against pasted robots.txt rules without uploading site configuration.
Uses specific user-agent groups and longest matching Allow or Disallow rule precedence.
✓ Dane wejściowe narzędzi są przetwarzane w Twojej przeglądarce.✓ Bez konta
Test crawler access against robots.txt rules locally.
When to use this robots.txt tester
Use it before deploying crawl rules or while investigating a blocked path.
- Check crawler access to a path.
- Verify an Allow exception beneath a blocked folder.
- Inspect wildcard and sitemap directives privately.
Powiązane narzędzia
Powiązane narzędzia
FAQ
Pytania
Pytania
How are conflicting rules resolved?
The most specific matching path wins, with Allow winning when matching rules have equal length.
Does this guarantee indexing?
No. robots.txt controls crawling; indexing also depends on other page and crawler signals.