Robots.txt Generator
Create a robots.txt with allow/disallow rules, AI crawler blocking and your sitemap.
How to use
- Pick defaults, list private paths and add your sitemap.
- Upload the file to the root of your domain (yourdomain.com/robots.txt).
FAQ
Does robots.txt hide pages?
No. It asks crawlers not to crawl. Use a noindex tag or a password to keep pages out of search.