Generate a valid robots.txt
Tell search engines and crawlers exactly what they may and may not crawl. Pick a quick preset — allow all or block all — or switch to custom rules to set the user-agent, disallow and allow paths, a crawl-delay and your sitemap URL.
Copy the result or download a ready-to-deploy robots.txt for the root of your domain. Everything is generated in your browser, so nothing is uploaded.