What this means
Your pages are heavy (large HTML, significant inline scripts, high token bloat), making them expensive for AI crawlers to fetch and process. Crawl Cost ≥ 70 is "High" — crawlers may deprioritise or skip your pages.
The three penalty components
- HTML size: <50KB = 10, <200KB = 30, <500KB = 50, ≥500KB = 70
- Inline scripts: >100KB = +15, >50KB = +10
- Token bloat: >30× ratio = +15, >15× = +10
Address whichever components contribute the most to your score.
How to fix it
- Reduce HTML size: Minify HTML output, remove comments and whitespace, eliminate inline CSS/JS.
- Externalise scripts: Move inline scripts to external files with
async or defer.
- Reduce token bloat: See Fix Token Bloat for detailed guidance.
- Optimise images: Use external image URLs rather than inline base64.
- Simplify navigation: Reduce mega-menu DOM complexity.
Target
Aim for Crawl Cost < 40 ("Low"). Keep pages under 200KB HTML with minimal inline scripts and a token bloat ratio under 15×.