What Is llms.txt? An Honest Guide to the AI Site File, Who Reads It, and Whether You Need One

What llms.txt is, in plain English

It is a text file written in Markdown that sits at /llms.txt on your domain and summarizes your site for large language models. Think of it as a hand-written table of contents for a machine reader that has limited time and limited memory.

The idea was proposed by Jeremy Howard on llmstxt.org, dated September 3, 2024. His reasoning is simple. Context windows are still too small to swallow most websites whole, and web pages wrap their actual information in navigation, ads and JavaScript. A clean Markdown summary saves an AI tool from guessing what matters.

The proposal has since gone to a second version, updated in 2026, which adds guidance on publishing clean Markdown copies of individual pages. The core idea has not changed: one short file, written for machines, that points to the pages a human expert would point to.

What llms.txt is not

• Not robots.txt. It does not allow or block crawlers. Writing Disallow: lines in it does nothing.
• Not a sitemap. A sitemap lists every URL you want indexed. This file lists the few that explain you best.
• Not a ranking factor. Google says so in writing, and no other major AI search product has said it uses the file to rank or cite sites.
• Not a training opt-in or opt-out. The proposal itself says it was expected to be useful mainly at inference time, when an AI tool is answering a question, rather than for training.

Who actually reads llms.txt in 2026

The honest answer: some AI agents and developer tools read it, Google Search does not, and the big consumer chat products have not publicly confirmed using other sites' files. Here is what each party has actually said or done, checked in October 2026.

Notice the pattern. The companies that publish the file are mostly publishing it for their own documentation, where developers point coding assistants at a docs site and ask questions. That is the use case where the file clearly earns its keep. For a dental practice, a roofer or a law firm, the payoff is smaller and less certain.

Our position for clients: publish a clean file because it is cheap and some agents use it. Never sell it, or buy it, as an AI ranking strategy.

How it compares with robots.txt and sitemap.xml

These three files sit side by side at your site root and do three different jobs. robots.txt sets access rules, sitemap.xml lists everything you want indexed, and the AI file gives a curated summary for AI readers.

If you want to control whether AI companies crawl your site, robots.txt is where that happens, using the user-agent names each company publishes. If you want AI tools to understand you once they arrive, the curated file is one small helper. The proposal itself says it complements robots.txt by giving context for the content you already allow.

Anatomy of a good file, with a worked example

A valid file needs only one thing, an H1 with your site or business name. Everything else is optional, but the useful ones follow the same order the proposal describes.

1. H1: the name of the business or project. This is the only required line.
2. Blockquote summary: one or two sentences that say what you do, for whom, and where.
3. Plain notes (no headings): facts an AI should get right. Service area, hours, licenses, what you do not do.
4. H2 link sections: lists of [Page name](URL): one-line note, grouped by topic.
5. An H2 called Optional: secondary links an agent can skip when it is short on context.

Worked example: a local HVAC company

Here is the kind of file we would write for a two-location home-services business. The business is fictional; the structure is exactly what we ship.

# Coastal Air Heating & Cooling

> Licensed HVAC repair, replacement and maintenance for homes and light commercial buildings in Broward and Palm Beach counties, Florida. 24/7 emergency service.

- Service area: Fort Lauderdale to West Palm Beach. We do not serve Miami-Dade.
- We service all major brands. We install heat pumps and ductless mini-splits.
- Phone: (555) 010-0000. Hours: office 7am-7pm, emergency line 24/7.

## Services
- [AC repair](https://example.com/ac-repair/): diagnostics, same-day repair, common failure causes
- [AC replacement](https://example.com/ac-replacement/): sizing, SEER2 ratings, financing, timeline
- [Maintenance plans](https://example.com/maintenance-plan/): what a tune-up includes, visit schedule

## Help and answers
- [How long should an AC last in Florida?](https://example.com/ac-lifespan/)
- [Repair or replace calculator](https://example.com/repair-or-replace/)

## Company
- [About and licenses](https://example.com/about/): owners, license numbers, insurance
- [Reviews](https://example.com/reviews/)
- [Contact and service area](https://example.com/contact/)

## Optional
- [Blog](https://example.com/blog/)
- [Careers](https://example.com/careers/)

That is around 25 lines. It tells a machine reader what the business does, where it works, what it refuses, and where the best answers live. A model summarizing this company has far less room to invent things.

The Markdown page copies (version 2 of the proposal)

The newer version of the proposal also suggests a clean Markdown copy of important pages at the same URL with .md added, plus a link tag pointing to it. This matters most for documentation sites. For most small business sites we skip it, because the HTML pages are already clean, fast and hand-coded, and maintaining a second copy of every page invites drift.

Should your business publish one? A decision table

Most businesses should publish a short version because it takes 30 minutes and has no downside when done cleanly. Whether you should invest more than that depends on what kind of site you run.

If someone quotes you a large fee for this file alone, or claims it will get you into ChatGPT answers, walk away. That is the 2026 version of selling meta keywords, a tag Google's own SEO Starter Guide says it does not use.

How to create the file in 30 minutes

Write it by hand from a short list of your best pages, save it as UTF-8 plain text, upload it to your site root, and confirm it loads with a 200 status. Here is the exact sequence we follow.

1. Pull your top pages (5 minutes). Open Search Console, sort pages by clicks for the last 12 months, and note the top 20. Add your core service pages, about, contact and any pricing-free FAQ pages that answer real questions.
2. Write the summary (5 minutes). One sentence: what you do, who for, where. Avoid slogans. A model needs facts, not taglines.
3. Write the facts block (5 minutes). Service area, hours, phone, licenses, brands, what you do not do. These are the details AI answers most often get wrong.
4. Group the links (10 minutes). Services, answers, company, optional. Give each a one-line note in plain language.
5. Publish and test (5 minutes). Upload to the root so the file is at /llms.txt. Then check the four things below.

The four-point publish check

• Status code: the exact URL /llms.txt returns 200, not a redirect. On WordPress, a rewrite rule or plugin can 301 it to /llms.txt/ with a trailing slash. Some clients will follow that, some will not.
• Content type: the server sends text/plain or text/markdown, not an HTML page.
• No login or firewall block: security plugins and bot-protection rules sometimes block unknown user agents. Test from outside your network.
• Facts match the site: phone, address and service area match your Google Business Profile and contact page exactly.

Running Chrome Lighthouse against your homepage will also flag a server error on the file, which is a quick sanity check after any hosting or plugin change.

Seven mistakes we see on audits

The most common failure is an auto-generated file that lists every URL on the site, followed by stale facts that contradict the rest of the web. Both make the file less useful than having none.

1. The URL dump. A plugin outputs 2,000 links in alphabetical order. That is a sitemap in the wrong format, and it defeats the one purpose of the file, which is curation.
2. Robots-style rules. Lines like Disallow: /private/ or User-agent: in the file. They do nothing. Access control belongs in robots.txt.
3. Stale facts. An old office address, a retired phone number, a service you dropped. AI tools may repeat whatever you hand them, so outdated facts spread.
4. Marketing copy instead of information. "The best roofer in Florida" tells a model nothing. "Licensed roofing contractor, residential re-roofs and repairs in Broward County" tells it everything.
5. Keyword stuffing. Repeating city names and service terms forty times. Nobody has shown this helps, and it reads as spam to any human reviewing it.
6. Linking pages that are blocked or noindexed. If robots.txt blocks a URL, pointing to it here sends a mixed message. Link only public, indexable pages.
7. Set and forget. The file is written once at launch and never touched. Put a quarterly review on the calendar alongside your technical SEO audit.

What moves AI visibility more than this file

Being cited by AI answers depends mostly on the same things that win in search: pages that can be crawled, clear facts about who you are, and content specific enough to quote. The AI text file sits near the bottom of the priority list.

Google's AI guide says its generative features run on its core Search systems, and it tells site owners to keep following SEO fundamentals and to create original, expert content. That lines up with what we see across client accounts. Here is the order we work in:

1. Crawlable, fast, indexable pages. If search engines cannot fetch and render a page, no AI answer built on search will find it. Our technical SEO checklist covers the full list.
2. Answer-first content. Each page opens with a direct answer, then the detail. This is the core of answer engine optimization.
3. Consistent entity facts. Name, address, phone, services and service area identical on your site, schema markup, Google Business Profile and directories.
4. Proof that others vouch for you. Reviews, local press, partner mentions. AI tools lean on sources beyond your own site.
5. Specific, first-hand content. Real processes, real numbers, real examples. Generic copy is easy for an AI to replace with anyone else's.
6. The AI site file. Last, quick, and clean.

For the bigger picture, our guides on AI SEO and how to rank in ChatGPT walk through each layer, and GEO vs SEO explains why most of the work overlaps.

How Unified handles it for clients

We write the file by hand for every site we build or manage, keep it short, and check it automatically along with the rest of the site's technical health. It is one line item inside our AI search optimization work, never a product of its own.

On a large build the curation matters more. When we launched HonestyInsure.com, the site had 144 pages and guides indexed and 31 state pages. Listing all of them would have buried the handful of guides that actually answer buyer questions. A curated file points agents to the hubs and the best explainers, and lets the hub pages do the rest. You can see how that site is structured in the HonestyInsure case study.

Inside UnifiedIQ, our own platform, every page is scored for SEO and technical issues, and the same monitoring that catches a broken sitemap catches a redirecting or erroring AI file. When a client changes a phone number or adds a location, the file gets updated in the same pass as the schema, the Google Business Profile and the contact page.

What we would do if we were you, this week

• Check whether yoursite.com/llms.txt exists and what it says. If a plugin made it, open it. You may be surprised.
• Replace any URL dump with a curated 15 to 40 link file using the template above.
• Confirm the facts block matches your Google Business Profile word for word.
• Then spend the rest of your AI budget on the pages themselves. That is where citations come from.

If you want a second set of eyes, our search team reviews it as part of every technical audit, and you can ask us for one.

Common questions, answered