llms.txt Generator

llms.txt is a community proposal by Jeremy Howard of Answer.AI (September 2024): a markdown file at /llms.txt that gives language models a concise, curated map of your site — one H1 with the site name, a blockquote summary, then H2 sections of links with one-line descriptions. The idea is sound: LLM context windows are finite, HTML is noisy, so hand the model a clean index.

Here's the part most generator tools won't tell you: almost nothing consumes it yet. Publishing adoption is real — roughly 10% of domains have one, including Anthropic, Stripe, Cloudflare and Vercel — but Ahrefs found that 97% of llms.txt files get zero AI-crawler requests, and Google has said plainly that it doesn't use the file; John Mueller compared it to the meta-keywords tag. No major AI engine has confirmed consuming it for ranking or retrieval.

So why offer the tool? Because the honest case still holds up: agentic and developer tooling increasingly fetches /llms.txt on demand when a user points an assistant at your site; it's a ready-made file to hand-feed your docs to ChatGPT or Claude; and it's a zero-cost hedge — if consumption arrives, you're already published. Just don't let anyone sell it to you as a ranking lever, because today it isn't one.

Most llms.txt generators pipe your pages through GPT and invent summaries. That has three problems: it's slower, it costs money (which is why those tools have paywalls and quotas), and the model can misdescribe your own product — in a file whose entire purpose is to describe your site accurately to other models.

This generator makes the opposite trade. We crawl up to 30 pages (reading your sitemap first, following links as a fallback, and respecting your robots.txt), then template the file from metadata you already wrote: page titles, meta descriptions, headings. Your words, in the llmstxt.org format, in seconds. Where a page has no meta description we say so instead of making one up — the editable preview exists precisely so you can fill those gaps with a sentence you actually stand behind.

The same philosophy runs through the whole file: legal and login pages are moved to an "Optional" section (the spec's marker for skippable context), sections are ordered by how much of your site they cover, and nothing is fabricated. A curated twenty-line llms.txt beats a machine-dumped two-hundred-line one — the spec's own reference examples are short.

Publishing llms.txt makes your site easier to hand to a model. It does not change what assistants say when a customer asks "what's the best tool for X?" — that answer comes from AI search indexes and live web retrieval. If you want to know whether ChatGPT, Perplexity and friends actually mention your brand today, run the free AI Visibility Checker — it probes grounded models with real prompts and shows you the raw answers.