Skip to content
SkySiteAudit

Website Crawler

Crawl a whole website: follow internal links and sitemaps, then find duplicate titles, broken links, redirect chains, orphan pages and more across every page.

Fully working

Create a free accountLog in

What it checks

  • Every page-level check from the SEO Checker
  • Duplicate titles, descriptions and content
  • Broken internal links and redirect chains site-wide
  • Orphan pages (sitemap URLs not linked internally)
  • Trailing-slash duplicates and canonical targets

Where the data comes from

SkySiteAudit's own crawler (SkySiteAuditBot) fetching public pages.

Limits — stated honestly

  • Up to 50 pages per crawl for unverified sites; up to the published fair-use limit for verified sites.
  • Respects robots.txt and Crawl-delay.
  • JavaScript-rendered links are not followed (HTML only); lab metrics run on the first 3 pages.
  • Fair use: 20 page scans per day without an account; 10 site crawls per day with a free account. Details

Need help reading the results? See the Website Crawler guide in the help center.