robots.txt generator
Build a robots.txt with allow/disallow and optional sitemap.
Runs in this tab. The file stays here.
robots.txt
This runs in your browser. Nothing is uploaded.
About robots.txt generator
Build a robots.txt with allow/disallow rules and an optional sitemap URL — you host the file.
robots.txt is still the polite sign on the door for crawlers. Generators that 'submit to Google for you' are doing a different job. This page builds the text file contents from allow/disallow lines and an optional Sitemap: entry so you can paste it at your site root.
It does not publish anything. It does not call Search Console. You download or copy the output and deploy it on your origin like any other static file. Rules are only hints; malicious bots ignore them.
Sitemap and llms.txt generators cover neighboring discovery files. Manifest generator is for PWA metadata, not crawler policy.
How to use robots.txt generator
- 1Set user-agent and allow/disallow paths as needed.
- 2Optionally add a sitemap URL.
- 3Copy robots.txt and host it at /robots.txt on your site.
What it won't do
- Static text generator — no crawl analytics.
- Cannot enforce access control.
Common questions
Does this robots.txt generator submit to Google?
No. It only creates file contents. Submission and inspection happen in your Search Console (or not at all).
Will robots.txt hide private pages from everyone?
No. It asks well-behaved crawlers to skip paths. Authentication and proper access control still matter.
Can I allow one bot and block others?
Yes — use separate user-agent blocks in the rules you configure before copying.
Related tools
Why this one doesn't upload your file
There is no server to upload to. This page is a static file, and the work happens in your browser using the same graphics and WebAssembly code that renders every other site you visit. Your file is read from disk into memory, processed, and handed back as a download. Once the page has loaded you can disconnect from the network entirely and it keeps working. The longer explanation