LLMS.txt Generator
Two modes: 1) Auto (enter a URL to generate llms.txt) 2) Manual (choose a template and customize).
We will try to detect robots.txt and sitemap automatically.
Fill URL and click Generate…
Project Configuration
AI Access Control Options
Fill details or choose a template, then click Generate…
What is llms.txt and why it matters
llms.txt is a simple text file published on your domain that tells AI models and LLM crawlers how to
interact with your content. It complements your robots.txt and sitemap.xml, describing
access preferences, crawl limits, attribution, licensing, and links to sitemaps or policies.
How to create an llms.txt for your site
- Enter your website URL in the Auto tab and click Generate.
- Optionally set profile, max depth, and crawl delay. Include sitemaps and policy URLs for clarity.
- Click Copy or Download to export the file as
llms.txt. - Upload
llms.txtto the site root (e.g.,https://example.com/llms.txt). - Verify accessibility and update when your policies or sitemaps change.
Best practices
- Be explicit: state
access,follow_links,include_metadata, andmax_depth. - Link your sitemaps: include
sitemap:lines for faster discovery. - Reference policies: add
policy_urlandcontent_licensewhere applicable. - Keep it short: avoid overly long lists; prefer concise rules.
- Use HTTPS: ensure all URLs are canonical and secure.
Example llms.txt
# llms.txt profile: standard access: public follow_links: true include_metadata: true max_depth: 5 crawl_delay: 0 site: https://example.com sitemap: https://example.com/sitemap.xml policy_url: https://example.com/privacy-policy content_license: CC BY 4.0 prompt_usage: allowed attribution_required: false
Where to place the file
Upload to your domain root so it’s available at https://yourdomain.com/llms.txt. For subdomains, publish
a separate file per subdomain if rules differ.
Frequently asked questions
No.
robots.txt targets web crawlers for indexing, while llms.txt communicates preferences for AI/LLM usage, attribution, and data handling.Place it at the domain root so it’s accessible at
/llms.txt. Use HTTPS and ensure it’s publicly readable.Yes. Adding
sitemap: lines helps AI agents find your content boundaries quickly.Use Standard for concise guidance. Choose Full when you need deeper crawling and more metadata.
Open
/llms.txt in a browser to confirm it loads. Keep it in sync with your robots.txt and sitemap.xml.