llms.txt
llms.txt is an emerging convention: a plain-text file at /llms.txt that tells AI crawlers which parts of your site are worth reading. Think of it as robots.txt inverted, guiding engines toward your best content rather than away.
Should you ship an llms.txt file?
Where robots.txt is about disallowing, llms.txt is about pointing. You list the highest-signal URLs on your site, such as product pages, comparison pages, founder and methodology pages, so an AI crawler with a limited budget spends it on the content that actually represents you instead of crawling pagination and tag archives.
The major AI crawlers have not all standardized on it yet, so it is not a guaranteed win today. But the cost of shipping one is close to zero, adoption is climbing, and the downside is nonexistent. It is a cheap option on a standard that is clearly forming.
A good llms.txt is also a useful forcing function: deciding which 30 URLs represent your business best is the same exercise as deciding where to focus your GEO work. GrowthManager generates and maintains llms.txt and llms-full.txt for every client site as part of the standard infrastructure setup.
Two file formats matter in practice. The short llms.txt is a markdown-style index that points to your highest-signal URLs with one-line descriptions, ideal for crawlers with a tight budget. The longer llms-full.txt embeds the actual content of those pages in plain text, so a retrieval-only crawler that respects the standard can ingest your story in a single fetch instead of orchestrating dozens of crawl cycles. Most sites ship both, refresh them weekly, and place them at the root with the standard /llms.txt and /llms-full.txt paths so detection requires no out-of-band discovery.
See where you stand
Free AI visibility check across ChatGPT, Perplexity, Gemini, and Claude. Results in under a minute.