robots.txt tells search-engine crawlers what they can index. sitemap.xml tells them what pages exist. llms.txt is the newer, less formal counterpart aimed at a different reader entirely: large language models and the AI assistants built on top of them.
What llms.txt actually contains
It's a plain markdown file — no XML, no special syntax — hosted at the root of a domain (e.g. /llms.txt) that gives a short, structured summary of a site: what it is, in one line; optional background context; and a set of linked sections pointing to the pages that matter most, each with a short description.
Why it exists separately from a sitemap
A sitemap lists every URL indiscriminately, which is exactly what a search-engine crawler wants but not particularly useful for a language model trying to understand a site's purpose and highlight its best pages within a limited context window. llms.txt is deliberately curated and short — a handful of the most important, most representative links rather than a full inventory.
Does it change how AI assistants answer questions right now?
Adoption and actual usage by AI providers is still evolving, and there's no confirmed guarantee any specific assistant reads or prioritizes it today. It costs very little to add, though, and it's a low-effort way to hand a clean, current summary to any crawler — human-built or AI-driven — that does choose to look for it.
What to put in it if you make one
Keep the top-level summary to one sentence. Group links under a few clear headings ("Docs," "Guides," "Product") rather than one long flat list, and write real one-line descriptions instead of just repeating the page title — the description is often more useful to a model than the link text alone.
TeckForge's llms.txt Generator builds the file section by section and downloads it ready to upload to your site's root — no markdown syntax to remember, no manual formatting to get wrong.
