The llms.txt File: What It Does, What It Doesn't, and What Actually Works
llms.txta plain-text file you add to your website's root directory that tells AI systems which pages are most important to read is a proposed open standard created by Jeremy Howard of Answer.AI in September 2024. It provides a Markdown-formatted guide that points AI models to your most important content, similar to how robots.txt tells search engine crawlers which pages to access. Over 600 websites have adopted it, including Anthropic, Stripe, Cloudflare, and Zapier. But no major AI platform has confirmed they read it, and Google's own engineers have compared it to the defunct keywords meta tag.
This article covers what llms.txt does, what the data says about whether it works, and what you should focus on instead if you want AI platforms to cite your business.
#What is llms.txt and why does it exist?
Most of a webpage's HTML is menus, tracking scripts, ads, and navigation. When an LLMlarge language model, the AI technology behind ChatGPT, Claude, and similar tools tries to read a website, it has to parse through all that noise to find the actual content. Jeremy Howard proposed llms.txt as a shortcut: a plain-text file at your website's root that lists your most important pages in Markdown format, so AI can skip the noise and read what matters.
Think of it this way. Robots.txta file that tells search engine crawlers which pages they're allowed or not allowed to visit controls access. Your sitemap helps with discovery. llms.txt is about curation. It says: "If you're an AI trying to understand my business, start here."
The file sits at yoursite.com/llms.txt and contains a title, a brief description, and links to key pages formatted in Markdown. Some sites also create an llms-full.txt with their complete documentation flattened into one file. Anthropic's version runs 8,364 tokens for the summary and 481,349 tokens for the full version.
#Who's using llms.txt?
The adoption pattern is specific. Developer tools, AI companies, and technical documentation sites moved first.
As of mid-2025, over 784 websites had implemented llms.txt files. The list includes Anthropic, Stripe, Cloudflare, Zapier, Perplexity, Cursor, Hugging Face, ElevenLabs, Yoast, and Solana. Adoption grew 600% between February and May 2025, from 15 to 105 sites in the Majestic Million dataset.
But the adoption is concentrated. An analysis of the top 1,000 most visited websites found a 0.3% adoption rate. Not a single major consumer platform (Google, Facebook, Amazon) has implemented it. The sites using llms.txt are technical companies building for developers, not the mainstream web.
#Does Google support llms.txt?
No. And they've been blunt about it.
Google's John Mueller compared llms.txt to the keywords meta tag, calling it "what a site-owner claims their site is about." He pointed out that LLMs don't request it, and you can see that in your server logs. Gary Illyes stated at Search Central Live in July 2025 that Google doesn't support llms.txt and isn't planning to.
The comparison to the keywords meta tag is pointed. In the 1990s, webmasters used the keywords meta tag to tell search engines what their page was about. Search engines stopped reading it because site owners would stuff it with whatever they wanted to rank for. Google's position: llms.txt has the same problem. It's self-reported. AI systems are better off reading your actual content than trusting a file that tells them what you claim your content is about.
#Do AI crawlers actually read llms.txt?
The data says they don't.
An independent audit from August to October 2025 checked server logs and found zero visits from GPTBot, ClaudeBot, PerplexityBot, or Google-Extended to llms.txt files. Traditional crawlers like Googlebot and Bingbot visited the file, but the AI-specific crawlers didn't.
A large-scale study across 300,000 domains found no measurable correlation between having an llms.txt file and receiving more AI citations. The researchers concluded that LLM traffic growth was driven by other factors, not the llms.txt file.
However, there's a nuance. When you paste an llms.txt URL directly into ChatGPT, Claude, or Perplexity, they read it and use it well. The file works as a reference document when explicitly provided. It just doesn't get discovered and read automatically by AI crawlers the way robots.txt gets read by Googlebot.
#Should you implement llms.txt anyway?
Yes, but with the right expectations.
The cost is near zero. It takes 30 minutes to create. It won't hurt anything. And the standard is early. If major AI platforms start reading it in 2027, you'll already have it in place.
I've been building for AI platforms since 2022, and the pattern with new standards is consistent: the first wave of adopters looks like they're wasting time. The second wave looks smart. The question is whether you want to spend 30 minutes now or scramble later.
But don't mistake implementing llms.txt for having an AI search strategy. That's the mistake most people make. They add the file, check the box, and think they're "optimized for AI." The file is a footnote. The strategy is what matters.
#What actually works for AI visibility?
The research is clear on what moves the needle. Here's what the data supports, ranked by impact:
Structured content with answer-first formatting. Pages where each section starts with a direct answer to a question get cited more than pages that bury the answer. Research from the Princeton GEO paper found that adding citations, statistics, and direct answers to content improved AI visibility by 30-40%. This is the highest-impact change you can make.
Schema markupcode added to your website that tells search engines and AI what your content means in a structured format. Pages with three or more schema types have a 13% higher likelihood of being cited by AI. FAQ schemacode that marks up your questions and answers so AI can read and extract them directly appears in 10.5% of cited pages. Properly implemented schema increases citation rates by up to 80% compared to equivalent pages without it.
Entity authority across multiple platforms. Brand mentions correlate 3x more with AI citations than backlinks. Your Google Business Profile, industry directories, review sites, and third-party mentions all feed AI's trust evaluation. A business that exists only on its own domain gives AI limited signals.
Content freshness. Content updated within 30 days earns 3.2x more AI citations than stale pages. 85% of AI Overview citations come from content published in the last two years. If your last blog post is from 2023, AI is likely citing your competitors instead.
llms.txt. Implement it. Spend 30 minutes. Then move on to the items above, which have data behind them.
#What should you implement first?
If you're starting from zero, here's the order that produces the fastest results:
1. Restructure your top 5 pages with question-based headings and answer-first paragraphs. This takes a day and has the highest citation impact. 2. Add FAQ schema to those same pages. JSON-LDa specific code format that search engines and AI prefer for reading structured information on your pages format. Pages with FAQ schema get cited at roughly twice the rate of pages without it. 3. Audit your directory listings for consistency. Name, address, phone, specialties should match across Google Business Profile, industry directories, and review platforms. 4. Publish fresh content that answers the questions your prospects ask. One well-structured article per week beats ten generic blog posts per month. 5. Add llms.txt. Point it to your most important pages. Takes 30 minutes. Low effort, low risk, potential future upside.
That order puts the highest-impact work first and the speculative work last.
I run AEO implementations for businesses and I've tested every tactic in this list. llms.txt gets attention because it's novel and it has a good origin story. But when I look at what moves citation rates for the businesses I work with, it's structured content and schema markup doing the heavy lifting. Not a text file. The businesses that win in AI search are the ones that make their actual content easy for AI to parse, not the ones that add a roadmap and hope AI follows it.
If your website doesn't have FAQ schema on its key pages, that's where I'd start. Not llms.txt.
#Frequently asked questions about llms.txt
Is llms.txt a replacement for robots.txt? No. They serve different purposes. Robots.txt controls which pages search engine crawlers can access. llms.txt curates which pages AI models should prioritize. They work alongside each other. Neither replaces the other.
How long does it take to create an llms.txt file? About 30 minutes for a standard business website. You write a title, a brief description of your business, and link to your 5-10 most important pages in Markdown format. Place it at your domain root (yoursite.com/llms.txt).
Will llms.txt help my business show up in ChatGPT? Current data shows no correlation between having llms.txt and receiving AI citations. What does correlate: structured content, schema markup, entity authority across directories, and content freshness. Implement llms.txt as a low-cost bet, but don't rely on it as your AI visibility strategy.
Should I wait for AI platforms to officially support llms.txt before adding it? No reason to wait. It costs 30 minutes and zero dollars. If platforms start reading it, you're ready. If they don't, you lost nothing. The risk is asymmetric in your favor.