What is llms.txt?
llms.txt is a small, machine-readable text file served from the root of a site (for example https://example.com/llms.txt) that communicates high-level guidance to AI systems and LLM-based assistants.
Unlike robots.txt which controls crawl behavior, llms.txt describes content intent: canonical sources, preferred excerpts, licensing, and pointers to documentation that help AI summarize and attribute content accurately.
Why it matters
As AI assistants grow more prevalent, sites that provide clear signals will be easier to summarize and cite correctly. llms.txt is an easy way to:
- Signal authoritative pages and preferred snippets.
- Provide license and contact info for reuse and attribution.
- Exclude private or low-value paths from being used as answers.
How it works
The file is plain UTF-8 text with simple directives. Parsers should be tolerant: ignore unknown directives and support comments. A typical parser reads the file, extracts fields, normalizes URLs, and applies precedence rules (site-level fields override page hints).
Common directives include:
Site:— the canonical site originCanonical:— a preferred URL for general summariesImportant-Page:— path + short label for priority pagesExcerpt:— one-sentence preferred summaryLicense:andContact:Disallowed:— paths not to use for answers
Example llms.txt
# llms.txt example Site: https://example.com Contact: https://example.com/contact License: https://example.com/terms#content-use Canonical: https://example.com/about Excerpt: ExampleCorp provides privacy-first AI search and site tools. Important-Page: /docs/getting-started/ — Getting started guide Important-Page: /pricing/ — Pricing details Prefer: structured-data Disallowed: /dashboard/ Sitemap: https://example.com/sitemap.xml Updated: 2026-03-15
Tip: Keep the file short and authoritative — AI systems prefer concise signals.
Where to place llms.txt
Publish at the host root: /llms.txt. Serve it as text/plain; charset=utf-8. Ensure no authentication is required if you want broad discoverability. Add a sitemap reference in /robots.txt to help crawlers find it.
How to create an effective llms.txt
Start small and iterate. A minimum viable llms.txt contains:
- Site — absolute origin.
- Canonical — your preferred human-facing summary URL.
- Excerpt — a short (1–2 sentence) canonical summary for assistants to prefer.
- Important-Page — 3–10 prioritized pages with short labels.
- License and Contact — links for reuse and questions.
Store the file in source control and add an automated check to ensure the Updated: date is refreshed when content changes.
Best practices
- Use absolute URLs for site-level fields; use paths for page-level directives.
- Keep excerpts neutral and representative — avoid marketing language.
- Prefer existing structured data on pages; use
Prefer: structured-datato hint parsers. - Do not include sensitive or private URLs — use
Disallowed:for low-value paths. - Test with a crawl/audit tool to ensure the file is reachable and parsed correctly.
Frequently asked questions
Is llms.txt required?
No. It is optional but helpful for sites that want clearer AI attribution and summaries.
Will search engines use it?
Search engines may not treat llms.txt the same way as robots.txt yet. Treat it as a complement to existing signals.
Ready to generate your llms.txt?
Use our generator to produce a well-formed llms.txt tailored to your site. You can copy, customise, and deploy the result in seconds.
Open LLMS.txt Generator — Create & Download