llms.txt Generator

llms.txt is a plain Markdown file placed at the root of a website, at /llms.txt, that gives large language models a short, curated guide to the site: what it is about and which pages are worth reading. The idea was proposed in 2024 by Jeremy Howard of Answer.AI and is documented on the project's own site, llmstxt.org.

The problem it targets is practical. A normal web page is built for browsers, so the text a model actually needs sits between navigation menus, cookie banners, scripts, footers and ads. When an AI assistant or agent fetches a page to answer a question, all of that noise competes for a limited context window. Site owners, on the other hand, already know which pages explain their product best. llms.txt lets them say so in one small file, with a sentence of context next to each link.

Think of it as a hand-written table of contents for machines. It does not list every URL the way a sitemap does, and it does not grant or block access the way robots.txt does. It simply points a reader that has already arrived toward the material you consider most useful.

Its status matters. llms.txt is a proposal, not a ratified web standard. No standards body maintains it, no AI company is obliged to read it, and support varies from tool to tool. Adopting it is voluntary and cheap: one text file, no code, no change to your existing pages. That low cost is why many documentation sites, developer tools and publishers have added one, and why it is worth understanding what the file can and cannot do before you write yours.

The format is ordinary Markdown with a fixed order. Here is a small file of the kind this generator produces, line by line:

• # Acme Docs
• > Acme is an open source job queue for Node.js with retries, scheduling and a web dashboard.
• All examples assume Node.js 20 or newer. The API reference is generated from the source code.
• ## Docs
• - [Quick start](/docs/start): install, first queue and first worker
• - [Retries](/docs/retries): backoff options and dead letter handling
• ## Optional
• - [Changelog](/changelog): release notes for every version

In a real file the links are full addresses on your own domain.

Reading from the top:

1. One H1 with the site or project name. It is the only required line, and the file should have exactly one.
2. A blockquote summary that starts with >. This is the line a model is most likely to read and repeat, so it should say plainly what the site is.
3. Optional detail paragraphs, plain text or lists with no headings. The generator removes leading # marks from any line you type here, so a stray heading cannot break the structure.
4. H2 sections, each holding a Markdown list in the form - [title](url): note. The note after the colon is optional but is where most of the value lives.
5. The Optional section, a reserved H2 titled exactly Optional. Links there are secondary, and an agent short on space may skip them. Because agents can simply cut the file off at that point, it belongs at the end.

The tool also tidies the output: extra spaces and line breaks inside names, summaries, titles and notes are collapsed to single spaces, and square brackets in link titles are escaped so they cannot break the link syntax. Sections with no title and no links are left out entirely.

The generator above builds the file from a form, so you never have to remember the syntax. It is free, needs no sign up, and everything happens in your browser: the tool does not crawl your site, fetch your pages or upload what you type.

1. Enter the site or project name. This becomes the H1 and is the only required field.
2. Write a one-line summary. One or two sentences on what the site is and who it is for.
3. Add details if needed. Short paragraphs on things a model should know before reading further, such as versions, licensing or how the docs are organised.
4. Build your link sections. Typical section titles are Docs, Guides, API Reference, Examples or Policies. For each link, fill in a title, the full URL and a note that says what the page covers. Use Add link and Add section as you go, and the Add Optional section button for secondary material such as changelogs or archives. That button disappears once a section called Optional exists.
5. Generate and read the Checks table. Errors mark lines that will come out malformed, such as a link without a URL or an address that is not a full http:// or https:// URL. Warnings are advice: a missing or overly long summary, an empty or duplicated section, or an Optional section that is not last.
6. Copy or download. The download is saved as llms.txt in plain text. If you change any field afterwards, the result clears so you never copy a stale version.

Finally, upload the file to the root of your domain so it is reachable at /llms.txt, next to robots.txt. If you are reviewing which AI bots may access the site at the same time, an AI crawler robots.txt generator covers that side of the job.

The format is easy; the curation is where files differ. An llms.txt that lists two hundred URLs with no notes is little better than a sitemap. One that lists the twenty pages that genuinely explain your product, each with a clear note, is something a model can use.

A few habits that make a difference:

• Keep the summary short. A sentence or two is enough. The generator warns once the summary passes 300 characters, because a long blockquote stops being a summary.
• Write notes that describe content, not marketing. "Rate limits, error codes and pagination for the REST API" helps an agent decide whether to open a page. "Everything you need to know" does not.
• Link to canonical, stable pages. Use full absolute URLs, avoid tracking parameters, and prefer addresses that will not move with the next redesign.
• Prefer pages that read cleanly. Reference docs, guides and plain explanatory pages work better than heavily scripted landing pages. If you publish Markdown versions of your docs, link to those.
• Group by purpose. Readers look for Docs, API Reference or Examples, not for your internal team names. Give each section a distinct title; the Checks table flags duplicates.
• Put secondary material in Optional. Changelogs, blog archives and legal pages are useful but rarely essential.
• Keep it short and current. Revisit the file when your site structure changes, a product is renamed or key pages move. A broken link in llms.txt points an agent straight to a dead end.

Treat the file as editorial work. It is your answer to the question "if someone could read only a handful of our pages, which should they be?"

It depends on what you expect from it. llms.txt is read when something chooses to read it, and today that is mostly tools acting on a specific request: AI coding assistants and agent frameworks that load a project's documentation, or a user who pastes the file into a chat to give a model the right context. Some documentation platforms generate the file automatically for the sites they host, and public directories track thousands of sites that publish one.

What it is not, as far as anyone can verify, is a ranking signal. Major search engines do not document using llms.txt for ranking, and there is no public evidence that publishing one increases traffic or makes AI assistants cite you more often. Any service that promises rankings or AI mentions from llms.txt alone is overselling it.

The honest case for adding one is modest and still reasonable. It costs a few minutes, it changes nothing about your existing pages, and for any agent that does read it, it replaces guesswork with a clear map written by the people who know the site best. For developer tools and documentation sites in particular, that map can noticeably improve the answers an assistant gives about your product.

llms.txt also works alongside the files you already have, not instead of them:

Access control stays in robots.txt; an AI crawler robots.txt generator can write rules for specific AI bots. Good, well-structured pages remain the foundation, since an llms.txt can only point to content that is worth reading.