What Is llms.txt? An Honest Guide to the AI Site File, Who Reads It, and Whether You Need One

What is llms.txt, in plain English?

It is a text file written in Markdown that sits at /llms.txt on your domain and summarizes your site for large language models. Think of it as a hand-written table of contents for a machine reader that has limited time and limited memory.

The idea was proposed by Jeremy Howard on llmstxt.org, dated September 3, 2024. His reasoning is simple. Context windows are still too small to swallow most websites whole, and web pages wrap their actual information in navigation, ads and JavaScript. A clean Markdown summary saves an AI tool from guessing what matters.

The proposal has since gone to a second version, updated in 2026, which adds guidance on publishing clean Markdown copies of individual pages. The core idea has not changed: one short file, written for machines, that points to the pages a human expert would point to.

What llms.txt is not

• Not robots.txt. It does not allow or block crawlers. Writing Disallow: lines in it does nothing.
• Not a sitemap. A sitemap lists every URL you want indexed. This file lists the few that explain you best.
• Not a ranking factor. Google says so in writing, and no other major AI search product has said it uses the file to rank or cite sites.
• Not a training opt-in or opt-out. The proposal itself says it was expected to be useful mainly at inference time, when an AI tool is answering a question, rather than for training.

Who actually reads llms.txt in 2026?

The honest answer: some AI agents and developer tools read it, Google Search does not, and the big consumer chat products have not publicly confirmed using other sites' files. Here is what each party has actually said or done, checked in October 2026.

Notice the pattern. The companies that publish the file are mostly publishing it for their own documentation, where developers point coding assistants at a docs site and ask questions. That is the use case where the file clearly earns its keep. The same logic applies to any business that wants an AI assistant to answer from its own material, which is the subject of our guide to building an AI knowledge base. For a dental practice, a roofer or a law firm, the payoff is smaller and less certain.

Our position for clients: publish a clean file because it is cheap and some agents use it. Never sell it, or buy it, as an AI ranking strategy.

How is llms.txt different from robots.txt and sitemap.xml?

These three files sit side by side at your site root and do three different jobs. robots.txt sets access rules, sitemap.xml lists everything you want indexed, and the AI file gives a curated summary for AI readers.

If you want to control whether AI companies crawl your site, robots.txt is where that happens, using the user-agent names each company publishes. If you want AI tools to understand you once they arrive, the curated file is one small helper. The proposal itself says it complements robots.txt by giving context for the content you already allow.

Which file or setting controls what

Owners often reach for llms.txt to do a job it cannot do. Use this table to pick the right control for the outcome you want.

Sources: OpenAI's crawler documentation, Google's Search generative AI control help page and its AI features documentation.

What goes in a good file? A worked example

A valid file needs only one thing, an H1 with your site or business name. Everything else is optional, but the useful ones follow the same order the proposal describes.

1. H1: the name of the business or project. This is the only required line.
2. Blockquote summary: one or two sentences that say what you do, for whom, and where.
3. Plain notes (no headings): facts an AI should get right. Service area, hours, licenses, what you do not do.
4. H2 link sections: lists of [Page name](URL): one-line note, grouped by topic.
5. An H2 called Optional: secondary links an agent can skip when it is short on context.

Worked example: a local HVAC company

Here is the kind of file we would write for a two-location home-services business. The business is fictional; the structure is exactly what we ship.

# Coastal Air Heating & Cooling

> Licensed HVAC repair, replacement and maintenance for homes and light commercial buildings in Broward and Palm Beach counties, Florida. 24/7 emergency service.

- Service area: Fort Lauderdale to West Palm Beach. We do not serve Miami-Dade.
- We service all major brands. We install heat pumps and ductless mini-splits.
- Phone: (555) 010-0000. Hours: office 7am-7pm, emergency line 24/7.

## Services
- [AC repair](https://example.com/ac-repair/): diagnostics, same-day repair, common failure causes
- [AC replacement](https://example.com/ac-replacement/): sizing, SEER2 ratings, financing, timeline
- [Maintenance plans](https://example.com/maintenance-plan/): what a tune-up includes, visit schedule

## Help and answers
- [How long should an AC last in Florida?](https://example.com/ac-lifespan/)
- [Repair or replace calculator](https://example.com/repair-or-replace/)

## Company
- [About and licenses](https://example.com/about/): owners, license numbers, insurance
- [Reviews](https://example.com/reviews/)
- [Contact and service area](https://example.com/contact/)

## Optional
- [Blog](https://example.com/blog/)
- [Careers](https://example.com/careers/)

That is around 25 lines. It tells a machine reader what the business does, where it works, what it refuses, and where the best answers live. A model summarizing this company has far less room to invent things.

The Markdown page copies (version 2 of the proposal)

The newer version of the proposal also suggests a clean Markdown copy of important pages at the same URL with .md added, plus a link tag pointing to it. This matters most for documentation sites. For most small business sites we skip it, because the HTML pages are already clean, fast and hand-coded, and maintaining a second copy of every page invites drift.

Should your business publish an llms.txt file?

Most businesses should publish a short version because it takes 30 minutes and has no downside when done cleanly. Whether you should invest more than that depends on what kind of site you run.

That last row is where the format is most useful for a growing business: when we build custom AI solutions that answer from a company's own documents, a short, curated index of the right sources does the same job internally that llms.txt does on the public web.

If someone quotes you a large fee for this file alone, or claims it will get you into ChatGPT answers, walk away. That is the 2026 version of selling meta keywords, a tag Google's own SEO Starter Guide says it does not use.

How do you create the file in 30 minutes?

Write it by hand from a short list of your best pages, save it as UTF-8 plain text, upload it to your site root, and confirm it loads with a 200 status. Here is the exact sequence we follow.

1. Pull your top pages (5 minutes). Open Search Console, sort pages by clicks for the last 12 months, and note the top 20. Add your core service pages, about, contact and any pricing-free FAQ pages that answer real questions.
2. Write the summary (5 minutes). One sentence: what you do, who for, where. Avoid slogans. A model needs facts, not taglines.
3. Write the facts block (5 minutes). Service area, hours, phone, licenses, brands, what you do not do. These are the details AI answers most often get wrong.
4. Group the links (10 minutes). Services, answers, company, optional. Give each a one-line note in plain language.
5. Publish and test (5 minutes). Upload to the root so the file is at /llms.txt. Then check the four things below.

The four-point publish check

• Status code: the exact URL /llms.txt returns 200, not a redirect. On WordPress, a rewrite rule or plugin can 301 it to /llms.txt/ with a trailing slash. Some clients will follow that, some will not.
• Content type: the server sends text/plain or text/markdown, not an HTML page.
• No login or firewall block: security plugins and bot-protection rules sometimes block unknown user agents. Test from outside your network.
• Facts match the site: phone, address and service area match your Google Business Profile and contact page exactly.

Running Chrome Lighthouse against your homepage will also flag a server error on the file, which is a quick sanity check after any hosting or plugin change.

How to see whether anything actually reads it

Your server access logs answer the question better than any vendor claim. Every request for the file is logged with the user agent that asked for it.

1. Download the raw access log from your host (cPanel, Plesk and most managed WordPress hosts offer it).
2. Search it for the file name. On a Mac or Linux machine: grep "llms.txt" access.log.
3. Group the hits by user agent and note which AI crawlers, assistants or coding tools appear, and how often.
4. Repeat monthly. If nothing requests it for months, that tells you how much effort it deserves on your site.

If you want help drafting the summary and facts block, a general assistant works well as a first pass; our guide to ChatGPT for small business covers how to prompt it without handing over private data. Then check every fact by hand.

What llms.txt mistakes do we see on audits?

Seven mistakes cover almost everything we find. The most common failure is an auto-generated file that lists every URL on the site, followed by stale facts that contradict the rest of the web. Both make the file less useful than having none.

1. The URL dump. A plugin outputs 2,000 links in alphabetical order. That is a sitemap in the wrong format, and it defeats the one purpose of the file, which is curation.
2. Robots-style rules. Lines like Disallow: /private/ or User-agent: in the file. They do nothing. Access control belongs in robots.txt.
3. Stale facts. An old office address, a retired phone number, a service you dropped. AI tools may repeat whatever you hand them, so outdated facts spread.
4. Marketing copy instead of information. "The best roofer in Florida" tells a model nothing. "Licensed roofing contractor, residential re-roofs and repairs in Broward County" tells it everything.
5. Keyword stuffing. Repeating city names and service terms forty times. Nobody has shown this helps, and it reads as spam to any human reviewing it. The same goes for auto-written summaries; our guide to AI content creation explains why an AI draft still needs a person to check it.
6. Linking pages that are blocked or noindexed. If robots.txt blocks a URL, pointing to it here sends a mixed message. Link only public, indexable pages.
7. Set and forget. The file is written once at launch and never touched. Put a quarterly review on the calendar alongside your technical SEO audit.

What moves AI visibility more than llms.txt?

Being cited by AI answers depends mostly on the same things that win in search: pages that can be crawled, clear facts about who you are, and content specific enough to quote. The AI text file sits near the bottom of the priority list.

Google's AI guide says its generative features run on its core Search systems, and it tells site owners to keep following SEO fundamentals and to create original, expert content. That lines up with what we see across client accounts. Here is the order we work in:

1. Crawlable, fast, indexable pages. If search engines cannot fetch and render a page, no AI answer built on search will find it. Our technical SEO checklist covers the full list.
2. Answer-first content. Each page opens with a direct answer, then the detail. This is the core of answer engine optimization.
3. Consistent entity facts. Name, address, phone, services and service area identical on your site, schema markup, Google Business Profile and directories.
4. Proof that others vouch for you. Reviews, local press, partner mentions. AI tools lean on sources beyond your own site.
5. Specific, first-hand content. Real processes, real numbers, real examples. Generic copy is easy for an AI to replace with anyone else's.
6. The AI site file. Last, quick, and clean.

If you are comparing tools that promise to generate or monitor the file for you, our review of AI marketing tools separates the ones that measure real citations from the ones that only produce reports.

For the bigger picture, our guides on AI SEO and how to rank in ChatGPT walk through each layer, and GEO vs SEO explains why most of the work overlaps.

How we handle llms.txt at Unified

We curate the file by hand, keep it short, and let our platform serve it with the rest of a site's SEO settings. It is one line item inside our AI search optimization work, never a product of its own.

Served by UnifiedIQ, curated by a person

On client sites running UnifiedIQ, our own platform, the file is served alongside titles, descriptions, sitemaps, redirects, breadcrumbs and connected schema for the business, its services and its people. That keeps it out of yet another plugin. A person still chooses the links and writes the facts block, and when a client changes a phone number or adds a location, we update the file in the same pass as the schema, the Google Business Profile and the contact page.

Curation on a large site

Curation matters most on large sites. Listing every page of a site with thousands of them, like the 3,066 pages we have live for Nusbaum Medical Centers, would bury the handful of guides that actually answer buyer questions. A curated file points agents to the hubs and the best explainers, and lets the hub pages do the rest.

Judged by citations, not by the file

Every week UnifiedIQ asks ChatGPT, Perplexity, Gemini, Claude and Google AI Overviews the questions a client's customers ask, and records who gets named. That weekly check, not the existence of a file, is how we judge whether AI visibility is improving. When a competitor is cited instead, the fix is almost always a better page, which the platform plans and our team edits and approves.

What we would do if we were you, this week

• Check whether yoursite.com/llms.txt exists and what it says. If a plugin made it, open it. You may be surprised.
• Replace any URL dump with a curated 15 to 40 link file using the template above.
• Confirm the facts block matches your Google Business Profile word for word.
• Then spend the rest of your AI budget on the pages themselves. That is where citations come from.

If you want a second set of eyes, our search team reviews the file as part of every technical SEO audit, and you can ask us for one.

Common questions, answered