CORPUSTool page
AI crawler checker for robots.txt and site access.
Corpus Plan checks robots.txt, named AI crawler rules, normal crawler access, and whether pages are readable without form interaction.
No account or email needed for the first report. Pro is $99/mo for weekly monitoring and up to 10 sites.
What gets checked
The audit reads robots.txt, then checks how the site responds to normal and AI-crawler user agents. It labels 429 and blocked responses carefully instead of guessing.
Why it sells the portal
Crawler rules change over time. Pro keeps watching the same site weekly and alerts you when a score drops.
What Pro adds
Pro generates robots rules for named AI crawlers alongside llms.txt, JSON-LD, FAQ suggestions, and the 30/60/90-day plan.
Turn the findings into your next steps
- Run the free audit. The report shows SEO, speed, trust, content, and AI-citability findings before signup.
- Claim the workspace. Keep the site, score history, and fix queue in one place.
- Go Pro when monitoring matters. Pro adds weekly re-scans, alerts, generated GEO files, grounded drafts, and traffic context.
Choose your next website check
- Search basics: Investigate a page missing from Google, then prioritize your SEO fixes.
- Business information: Check your schema and use the schema review checklist.
- AI search: Understand SEO versus GEO, check crawler access, and read what llms.txt can and cannot do.
- Customer inquiries: Test the path from a visit to an inquiry.
Corpus Plan does not promise rankings, traffic, revenue, or AI mentions. It shows what the checks found and keeps the fix work organized.