📄 llms.txt
Standard markdown directory file listing your pages, topics, and descriptions in the format AI crawlers expect.
llms.txt Generator & Validator
Generate a perfectly formatted llms.txt and llms-full.txt for your site. Help AI crawlers like GPTBot, ClaudeBot, and PerplexityBot understand your content structure, topics, and entity relationships.
How it works
Three steps to AI-readable content — no coding required.
Enter your domain
Paste any URL. We'll automatically discover your sitemap, crawl your pages, and understand your site structure.
We crawl & extract
SEODiff strips HTML boilerplate, extracts core content, identifies topics via TF-IDF analysis, and maps your site's entity structure.
Download your llms.txt
Get a perfectly formatted markdown file ready to deploy at /llms.txt. Pro users get full-site llms-full.txt with entity graphs and schema recommendations.
What you get
📋 llms-full.txt
Extended version with full content from each page. Requires domain ownership verification (DNS TXT record) to protect copyright compliance.
🔍 Topic Extraction
TF-IDF analysis identifies your site's core topics so AI systems understand your domain expertise.
🤖 robots.txt Check
Detect if GPTBot, ClaudeBot, or other AI crawlers are blocked by your robots.txt rules.
✅ Validator
Check your existing llms.txt for formatting errors, missing titles, and incomplete resource links.
🏷️ Schema Ready
Pro users get schema.org recommendations to pair with llms.txt for maximum AI visibility.
Related tools
Pair llms.txt with these tools for complete AI visibility.
Benchmark on Radar
See how your tech stack and industry compare, then ship llms.txt + schema to close the gap.