Utilities
robots.txt Tester
Paste a robots.txt file and test paths for Googlebot, Bingbot and AI crawlers like GPTBot. Runs in your browser. Then open a Vaneform site overview for the domain.
Verdict
Allowed · Matched rule: Allow: /
Known crawlers
- Allowed
- Disallowed
- Disallowed
- Disallowed
- Disallowed
- Disallowed
- Disallowed
- Disallowed
- Disallowed
- Disallowed
- Disallowed
Sitemaps
- https://example.com/sitemap.xml
How matching works
The most specific matching Allow or Disallow wins (longer path pattern). An empty Disallow allows everything for that group. A missing User-agent group falls back to * or to default allow. This tester follows common Google/Bing path rules and does not fetch live robots.txt.