What is llms.txt?
llms.txt is a proposed plain-text / Markdown convention placed at the root of a website. It offers a curated overview of the most important pages, in a format that is easy for people and machines to read.
- It is a summary, not a replacement for your website.
- It does not replace robots.txt or sitemap.xml.
- It does not guarantee crawling, citations, mentions or rankings.
Why was llms.txt proposed?
Websites can contain thousands of pages. Important content — products, documentation, pricing, guides — is often buried behind navigation and paginated lists. A curated file provides a compact, descriptive summary that points to the pages a reader (or an AI system) should look at first.
Adoption varies across AI systems and cannot be assumed.
Who should use llms.txt?
- SaaS websites with product, docs and pricing pages
- Documentation platforms and developer portals
- Content publishers and blogs
- Free-tool websites
- Agencies and consulting practices
- Ecommerce brands with catalog and policy pages
- Educational sites and large resource libraries
Very small websites may only need a compact file — one section is enough.
Benefits and limitations
- · Curated overview of the site
- · Clear page hierarchy
- · Concise page descriptions
- · Easy discovery of core resources
- · Better internal documentation
- · Human-readable website index
- · Proposed standard, not universally supported
- · No ranking guarantee
- · Does not override robots.txt
- · Does not force AI systems to cite a site
- · Requires maintenance
- · Can become noisy when overly long
What the Reflyma Generator creates
- Factual brand title and short summary
- One or two overview paragraphs
- Semantic categories (Product, Docs, Blog, Pricing, Legal…)
- Important page selection with a per-page score
- Page-level descriptions with confidence labels
- Optional resources section
- File metadata, token estimate and quality score
- Validation report and editable versions
Claims are checked against page evidence. Duplicate pages are collapsed, acronyms are normalized, and private routes are excluded.
Where to publish your llms.txt file
- Publish at the website root — for example
https://example.com/llms.txt. - Serve as plain text or Markdown-compatible text with a 200 response.
- Keep it publicly accessible with a stable URL.
- Update it whenever important website content changes.
- Do not block it in robots.txt unless intentional.
Automatic discovery by all providers cannot be assumed.
Professional validation checklist
- Brand title exists
- Summary is factual
- Important pages included
- Private pages excluded
- URLs resolve
- Canonicals used
- Descriptions present
- Sections are meaningful
- Duplicates removed
- Acronyms formatted
- File is not excessively large
- Claims are evidence-backed
- Legal pages included where useful
- Optional resources separated