SeenRank

SeenRank Blog

llms.txt: what it is, who reads it, do you need one

Updated 2026-05-13. By the SeenRank team.

Short answer: llms.txt is a proposed standard for a markdown file at https://yourdomain.com/llms.txt that gives AI agents a curated map of your most important content. As of May 2026, no major AI engine has publicly committed to reading or acting on it in production. Some developer-tooling AI agents are starting to fetch it during inference. The cost to publish one is roughly 30 minutes once. The honest answer to "do I need one" is: ship one anyway, it's a near-zero cost hedge on a standard that may matter in 6-12 months.

What is llms.txt, exactly?

llms.txt is a proposed text-file convention, modeled on robots.txt, designed to help AI agents and large language models find and prioritize your site's most important content. The original proposal came from Jeremy Howard (Answer.AI, fast.ai) in September 2024 and has been picked up by a growing number of developer-tooling and documentation sites since.

Structurally, llms.txt is a markdown file at the root of your domain (https://yourdomain.com/llms.txt) that follows a standardized format: one paragraph describing what your site is, then sections of bulleted links to your most important pages. Each link includes a one-line description so the AI agent knows what the page covers without having to fetch it first.

Two related files are sometimes also published:

  • llms-full.txt – a single markdown file containing the full text of every page on your site, concatenated. Useful for docs-heavy sites where AI agents in coding contexts want to ingest the full corpus.
  • llms.md – same idea as llms.txt but with a .md extension for sites that prefer it. Both are widely accepted.

The format, with a copy-paste template

The format is loose by design. The canonical structure looks like this:

# Site Name

> One-paragraph summary of what this site is, who it serves, and what kind
> of content lives here. Should be the kind of thing an AI agent can quote
> verbatim when describing your site.

## Documentation

- [Getting Started](https://example.com/getting-started.md): Installation and first steps
- [API Reference](https://example.com/api.md): Full endpoint documentation
- [SDK Guide](https://example.com/sdk.md): Language-specific SDKs and code examples

## Guides

- [Local SEO Playbook](https://example.com/local-seo.md): GBP-first approach for service businesses
- [GEO Best Practices](https://example.com/geo.md): How to win AI citations

## Optional

- [Changelog](https://example.com/changelog.md): Recent updates
- [About](https://example.com/about.md): Team and company background

Key conventions:

  • The site description goes in a blockquote right after the H1. This is what AI agents are most likely to quote when describing your site.
  • Group links into H2 sections that reflect how an AI agent would think about your content (Documentation, Guides, Tools, Pricing, etc.).
  • Each link includes a one-line description separated from the link by a colon. Brief, factual, no marketing fluff.
  • The "Optional" section at the bottom is where deprioritized but still-relevant pages go. AI agents are expected to skip these when surfacing primary content.
  • Linked URLs often have .md versions when possible. If your CMS supports it, serving a markdown variant of each page at example.com/page.md alongside example.com/page is the strongest signal.

Who actually reads llms.txt today?

Honest read on adoption as of May 2026:

AI engines that have publicly committed: none of the majors

Neither OpenAI, Anthropic, Google, Perplexity, nor Microsoft has publicly committed to reading or acting on llms.txt as a ranking signal in production. The major engines' web crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended) continue to fetch sites the same way they always have: through normal HTML pages, sitemap.xml, and robots.txt.

Developer-tooling AI agents: yes, increasingly

Cursor, Continue, Aider, Claude Code, and other AI coding assistants have been observed fetching llms.txt and llms-full.txt during inference when working with code that touches a documented library or framework. Anthropic, Stripe, Cloudflare, Supabase, and Vercel all publish llms.txt files specifically aimed at this audience.

Custom AI agents and GPTs: yes, if configured

Any custom GPT, Claude Project, or LangChain agent can be configured to fetch llms.txt as part of its system prompt or retrieval pipeline. Adoption here is patchy but growing, especially in B2B contexts where a company wants its own knowledge base structured for AI consumption.

Future: probably

The cost to AI engines of supporting llms.txt is low. The signal it provides (a curated, structured, machine-readable map of a site's most important content) is exactly the kind of thing retrieval-augmented generation systems benefit from. Best guess: by mid-2027, at least one major AI engine will publicly weight llms.txt as a discovery signal. Could be wrong, but the directionality is clear.

Do you need one in 2026? The honest answer

Three scenarios, three different answers.

Yes, immediately: developer-facing companies, docs-heavy sites, API products

Your audience uses AI coding tools that already fetch llms.txt. Publishing one improves how those tools represent your product to developers actively building with it. This is the only segment where there's measurable benefit today.

Probably yes, low-cost hedge: any content-driven business

SaaS, B2B services, agencies, consultants, content products. The cost is 30 minutes once, the benefit is uncertain in the near term but plausible in the medium term. The downside of not having one if the standard takes off is bigger than the cost of having one if it doesn't. Ship it.

Probably not yet: local services, e-commerce, B2C transactional

If your customers find you via Google Maps, Instagram, or word-of-mouth, llms.txt is unlikely to move your revenue in the next 12 months. There are higher-leverage moves (your basic GEO checklist) to do first. Come back to llms.txt in 2027.

How to ship one in 30 minutes

  1. Step 1: list your site's 5-15 most important pages

    Homepage, key product pages, primary documentation, top blog posts, pricing, about. Group them by what an AI agent would naturally think about (Documentation, Guides, Pricing, Tools).

  2. Step 2: write a 2-3 sentence site summary

    What is your site, who is it for, what's the main thing you do. Write it the way you'd want an AI agent to describe you when summarizing your site to a user.

  3. Step 3: paste into the template above

    Replace the example links with yours. One line per link, with a brief factual description.

  4. Step 4: save as llms.txt at your site root

    For most static sites: drop the file in your public/ folder. For React/Vite sites: client/public/llms.txt. For Hostinger: upload to public_html/. Verify it serves correctly at https://yourdomain.com/llms.txt.

  5. Step 5 (optional): add a reference to it in sitemap.xml

    Some practitioners add an entry to sitemap.xml pointing at /llms.txt so search engine crawlers see it consistently. Low-effort, no documented downside.

SeenRank publishes its own llms.txt at https://seenrank.com/llms.txt. Look at it for a working production example tuned to a B2B SaaS context.

Common llms.txt mistakes

  • Listing every page on the site. Defeats the point. The file is supposed to be curated. List your 10-20 most important pages, not your full sitemap.
  • Marketing-speak descriptions. "Industry-leading platform for next-generation X" is useless to an AI agent. Use factual, scannable descriptions: "Pricing tiers, feature comparison, billing FAQ."
  • Stale content. If your llms.txt links to a page that no longer exists or has moved, you're sending AI agents to a 404. Treat llms.txt as content; review quarterly.
  • Mixing your llms.txt with your robots.txt strategy. They're separate files with separate purposes. robots.txt tells crawlers what they can fetch; llms.txt tells AI agents what's worth fetching first.

Start with a free check

llms.txt is a low-priority hedge. The high-leverage GEO work (per-page first-paragraph audits, statistics, schema) drives 80%+ of your visibility gains. The free SeenRank check tells you which of those is missing in 30 seconds.

Run a free SeenRank check →

FAQ

Will having an llms.txt make ChatGPT cite me more?

Not directly, not yet. ChatGPT does not (as of May 2026) read llms.txt as a ranking signal. But if your audience uses ChatGPT plus a custom GPT or coding assistant configured to fetch llms.txt, you can be cited more in those contexts.

Does llms.txt replace sitemap.xml or robots.txt?

No. Different files, different purposes. sitemap.xml lists every URL for search engine crawlers. robots.txt grants or denies crawler access. llms.txt is a curated guide for AI agents. Publish all three.

Should I publish llms-full.txt too?

If you have well-structured documentation that benefits from being ingested wholesale by AI coding tools, yes. For typical marketing sites, no - it's mostly noise for that audience.

Can I publish llms.txt as just a list of URLs without descriptions?

Technically yes, but AI agents that do read llms.txt rely on the descriptions to decide which links are most relevant. A URL list alone provides much less signal than a properly formatted file.

How often should I update llms.txt?

When your important content list materially changes. Most sites update it every 1-3 months as new flagship pages ship or old ones get retired. Quarterly review is enough for most.

Run a free SeenRank check now →

Related: The 2026 GEO checklist  ·  What is generative engine optimization?  ·  AI Search Visibility: the 2026 guide