Definition
Blocklist Risk (0–5) is a simple count of how many of the five major AI crawlers (GPTBot, ClaudeBot, CCBot, Google-Extended, Googlebot) are blocked by your site, whether via robots.txt rules, WAF challenges, or bot management systems.
Risk levels
- 0 — None: All AI crawlers can access your site.
- 1 — Low: One bot blocked. Minor risk, but you're missing one AI pipeline.
- 2 — Medium: Two bots blocked. You're excluded from multiple AI systems.
- 3 — High: Three bots blocked. Significant AI visibility loss.
- 4–5 — Critical: Most or all AI crawlers blocked. Your content is effectively invisible to AI.
Impact on AI Readiness Score
Blocklist Risk triggers hard caps on the overall AI Readiness Score:
- If both GPTBot and ClaudeBot are blocked → AI Readiness capped at 15.
- If a WAF/CAPTCHA challenge is served → AI Readiness capped at 10.
These caps override the weighted score because no amount of good structure or schema can compensate for being completely inaccessible.
Common blocking mechanisms
- robots.txt Disallow — Explicit rules like
User-agent: GPTBot / Disallow: /
- WAF / Bot Management — Cloudflare Bot Management, Akamai, Imperva blocking unknown user-agents
- CAPTCHA challenges — Challenge pages served instead of content
- IP-based blocking — Blocking known AI crawler IP ranges