开发者可以使用

Robots.txt Generator

Build a robots.txt file with user-agent, allow, disallow, crawl-delay, and sitemap rules, then download it locally.

TODO-i18n: this tool's instructions and FAQs are currently available in English.

交互式工具区

无需注册账号

Inputs and generated outputs stay in this browser. Files and secrets are not uploaded to SiteIndex.

Rules (one per line)Sitemap URL (optional)

Robots.txt Generator是什么？

Build a robots.txt file with user-agent, allow, disallow, crawl-delay, and sitemap rules, then download it locally.

工作原理

Build a robots.txt file with user-agent, allow, disallow, crawl-delay, and sitemap rules, then download it locally.

常见用途

Build a robots.txt file with user-agent, allow, disallow, crawl-delay, and sitemap rules, then download it locally.

使用限制

Build a robots.txt file with user-agent, allow, disallow, crawl-delay, and sitemap rules, then download it locally.

常见问题

Robots.txt Generator是什么？: Build a robots.txt file with user-agent, allow, disallow, crawl-delay, and sitemap rules, then download it locally.