robots.txt is available

Confirmed Quick fixcrawl.robots_txt

Module
Crawlability & AI-bot access
Severity
Warning
Applies to
All pages

How to fix it

Serve a valid robots.txt with status 200.

Evidence

Confirmed. Documented by the search engine or AI operator itself.

How it counts

Read the full methodology.

Test your page

Runs the Crawlability & AI-bot access module, including this check.

Related checks