📄 llms.txt
Standard markdown directory file listing your pages, topics, and descriptions in the format AI crawlers expect.
llms.txt Generator & Validator
Generate a perfectly formatted llms.txt and llms-full.txt for your site. Help AI crawlers like GPTBot, ClaudeBot, and PerplexityBot understand your content structure, topics, and entity relationships.
How it works
Three steps to AI-readable content — no coding required.
Enter your domain
Paste any URL. We'll automatically discover your sitemap, crawl your pages, and understand your site structure.
We crawl & extract
SEODiff strips HTML boilerplate, extracts core content, identifies topics via TF-IDF analysis, and maps your site's entity structure.
Download your llms.txt
Get a perfectly formatted markdown file ready to deploy at /llms.txt. Pro users get full-site llms-full.txt with entity graphs and schema recommendations.
What you get
📋 llms-full.txt
Extended version with full content from each page. Requires domain ownership verification (DNS TXT record) to protect copyright compliance.
🔍 Topic Extraction
TF-IDF analysis identifies your site's core topics so AI systems understand your domain expertise.
🤖 robots.txt Check
Detect if GPTBot, ClaudeBot, or other AI crawlers are blocked by your robots.txt rules.
✅ Validator
Check your existing llms.txt for formatting errors, missing titles, and incomplete resource links.
🏷️ Schema Ready
Pro users get schema.org recommendations to pair with llms.txt for maximum AI visibility.
Related tools
Pair llms.txt with these tools for complete AI visibility.