What should I include in an llms.txt file?
Include a short site description and a deliberate set of useful source links, with labels that explain what each page covers. In Agent Gap's 6 October 2026 collection, pages were found in all 210 initial tasks, while 163 reached their defined endpoint. That difference helps explain why a list of discoverable URLs needs meaningful destinations; the study did not test llms.txt.
What we measured
Measured. The Agent Gap method used the ChatGPT Work browser interface for three public website tasks across 70 software vendors. Its 210 initial sessions recorded finding, correctness and endpoint reach separately; 30 selected repeat sessions were additional observations. Finding a page was not evidence of a completed transaction or of crawler-file use.
Reported. The llms.txt v2 proposal, read 28 October 2026, specifies a site or project H1, a possible summary and grouped Markdown links with descriptive notes. The file may cover the site root or a path beneath it. The H1 is its only required section; the remaining structure helps a reader choose sources.
What this means for you
Reasoned. Start with the reader's task, then select the page that actually resolves it. A pricing link should explain whether the destination contains public prices, a method link should identify the study it governs, and a product page should retain its product boundary. Repeating a page title gives less help when several destinations have similar names.
Keep important qualifications near the relevant link. A historical study needs its date and scope; an archived offer needs a historical label. Avoid making a short description more confident than the page it points to. The file is another publication of those facts, so a mistaken summary creates a second correction to maintain.
Choose a small set that you can keep accurate. Organise essential company facts, product information and research into useful groups rather than copying every navigation item. Keep supporting evidence behind the appropriate link instead of compressing a long factual claim into an unexplained heading.
Use existing approved descriptions where they remain true, including client material that the publisher has permission to show. An indexing exercise is not permission to rewrite those relationships. Each label should lead to the same factual meaning a person will find on the page.
- For a site owner: name the scope of each selected destination.
- For an editor: keep dates and limits when shortening a source description.
- For an agency: agree which approved pages belong in the delivered file.
Also asked as
What does an llms.txt example show?
Related
- Agent Gap's public website tasks
- Source evidence rules
- Generate from existing pages
- Check the served file
Next step
Use the Free 10-question check (one engine). No email is needed; runs are limited per network and per day, and by the site's daily allowance. Keep the answer and the source it cites beside any file-content decision.
For one approved source description that needs a factual page correction, Fix one answer, $490.
Our answer-study method, v1.1.
Prepared by the Broadcastwell team.
Page owner: Rohan Iyer, AI Engineer. Corrections: hello@broadcastwell.com