aicrawlable

What is llms.txt?

llms.txt is a proposed standard (from Jeremy Howard / Answer.AI) for a Markdown file at your site root — /llms.txt — that gives large language models a curated, machine-friendly map of your site.

The format

The spec defines a strict ordering:

# Project Name          ← required (the only required line)

> One-sentence summary  ← optional blockquote

Optional intro text (no headings).

## Section name         ← optional H2 sections
- [Page title](https://example.com/page): short description
- [Another page](https://example.com/other)

## Optional
- [Less important page](https://example.com/extra)

The ## Optional section has special meaning: those links can be skipped when context is limited.

llms.txt vs llms-full.txt

  • llms.txt — a concise index: links + descriptions, like a curated sitemap.
  • llms-full.txt — the full text of your pages concatenated into one file, so an LLM can ingest everything in a single fetch.

Where to put it

Upload the file to your web root so it's served at https://yourdomain.com/llms.txt.

Do AI crawlers read it?

Adoption is still early — as of 2026 most major AI crawlers don't reliably fetch llms.txt yet. Treat it as low-effort future-proofing and as a ready-made context file for your own RAG or documentation search. See our FAQ for the honest details.

Generate yours in seconds

Paste your URL and get both files, free.

Open the generator