llms.txt is a 2024 proposal by Jeremy Howard (Answer.AI): a Markdown text file at the address /llms.txt that works as a curated map of the site for language models.
What it looks like
It opens with a heading with the site’s name, continues with a short summary and then lists, by section, links to the main resources with a one-line description: documentation, pricing, guides, official facts pages. Some sites add llms-full.txt, with the full text of the pages. The difference from robots.txt is clear: robots.txt tells crawlers where they may go, llms.txt suggests what to read first.
Is it worth it?
It isn’t an approved standard. Google states explicitly that its Search doesn’t use it, and that it isn’t needed to appear in AI Overviews or AI Mode. For other engines and AI agents it can still be a cheap way to offer the right facts in an easy format. It costs little to prepare, as long as it stays in line with the pages.
Ours is at refinea.io/llms.txt. To see which bots read your site, see AI crawlers.