What does this robots.txt checker test?
It fetches the public robots.txt response, reviews its crawler directives, and checks whether it points search engines to a sitemap.
Public evidence
The tool reads the public response and shows what it measured.
No account required
Use the free check before deciding whether a full audit is useful.
Next step
Move from one signal to a complete SEO, GEO, and AEO audit.
What this tool checks
A focused answer from the public website response.
Check whether robots.txt is reachable, readable, and connected to your sitemap.
robots.txt HTTP response
Crawler directives and syntax
Declared sitemap locations
What you get
Enough evidence to choose the next step.
See whether robots.txt can be fetched
Find broad crawler blocks
Confirm sitemap discovery
FAQ
Can robots.txt remove a page from Google?
Robots.txt controls crawling, not guaranteed removal from an index. Pages that must not appear in search normally need an appropriate noindex response that crawlers can access.
Related free checks
Check the connected signals.
llms.txt Checker
Check whether a website publishes a reachable, readable llms.txt content index.
Run checkSitemap Validator
Check sitemap discovery and validate the XML or text response delivered by your website.
Run checkAI Crawler Checker
Review explicit robots.txt access rules for major AI and search crawlers.
Run check