Robots.txt generator
Create a robots.txt file with user agents, allowed or blocked paths and sitemap URLs. Review the rules before copying or downloading the file.
Fill one User-agent group with Allow/Disallow paths, one per line, and optional Sitemap. Download and review before replacing an existing file.
Result
Your result will appear here.
We count which tools are used most, without including entered values or file contents in the statistics. Privacy
How to use it
Fill one User-agent group with Allow/Disallow paths, one per line, and optional Sitemap. Download and review before replacing an existing file.
Method and limits
robots.txt controls crawling: it does not protect data or guarantee removal from search. Install at the domain root and review the rules.