Skip to content
SkySiteAudit

Robots.txt & Sitemap Validator

Fetch and parse robots.txt, test whether URLs are allowed, discover sitemaps (including sitemap indexes) and validate their structure.

Fully working

What it checks

  • robots.txt status, unknown directives, Crawl-delay, Sitemap lines
  • Allow/Disallow test for any path
  • Sitemap XML syntax, index files, invalid and cross-host URLs

Where the data comes from

Live fetch of /robots.txt and sitemap files.

Limits — stated honestly

  • Up to 10 sitemap files and 5,000 URLs are read per check.
  • Fair use: 20 page scans per day without an account; 10 site crawls per day with a free account. Details

Need help reading the results? See the Robots.txt & Sitemap Validator guide in the help center.