llms.txt

llms.txt is a plain-text file at the root of a site — example.com/llms.txt — that describes, in ordinary prose, what the site is and which pages matter.

It is the smallest possible thing you can do for an assistant, and it takes an afternoon.

What goes in it

A short description of the business, and a list of the pages worth reading, each with a line saying what it is. Not every page. The ones that answer questions.

Where adoption actually is

Across the fleet we crawled, a minority of sites carry an llms.txt at all, and a smaller minority of those advertise tools or context in it. Both counts are in the table below, rendered from the crawl rather than typed here — and as with WebMCP, it is overwhelmingly platform-shipped, Wix and Shopify, rather than written by the business.

Source — fleet crawl, 2026-08-28.

Its limits, stated

llms.txt is a convention, not a standard. No specification body has ratified it, no browser enforces it, and an assistant is free to ignore it. It is a courtesy that costs almost nothing and occasionally helps.

Treat it as the floor, not the work. A site with a beautiful llms.txt and a blank first response is still invisible.

SNAPSHOT 2026-09-04T01:47:53Z · measured 2026-09-03T03:25:55Z · bq query --dry_run then --nouse_legacy_sql over agent_ready.crawl_v1 (frozen, 27,112 rows) · rate = invisible / JUDGED, where judged excludes UNREACHABLE and CRAWLER_ERROR (agent_invisible IS NULL, tools/fleet_crawl.py:342) · receipt receipts/crawl_v1-census.json