llms.txt example: setup and testing guide

This llms.txt example gives agents a short, factual map of your website. You publish the Markdown file at /llms.txt, link the pages that answer common questions, and test the deployed response like any other release asset. The file can help an agent find useful source material, but it does not control crawlers or guarantee citations in an AI answer.

The current llms.txt proposal calls for an H1 with the site or project name. It also defines optional sections for a summary, context, and lists of links. Treat the format as an emerging convention. Keep your normal HTML, crawl controls, metadata, and structured data in place.

llms.txt example you can adapt

Start with a file that a developer can review in one screen:

# Acme API

> Acme API helps development teams validate shipping addresses and delivery zones.

Use the production API for customer requests. The sandbox uses separate credentials and test data.

## Start here

- [Quickstart](https://docs.example.com/quickstart.md): Create a sandbox key and send the first request.
- [API reference](https://docs.example.com/api.md): Review endpoints, fields, status codes, and limits.

## Policies

- [Pricing](https://example.com/pricing): Compare plans and current request limits.
- [Security](https://example.com/security): Review data handling and security controls.

## Optional

- [Changelog](https://docs.example.com/changelog.md): Check recent API changes.

The proposal requires only the H1. A useful llms.txt template adds a short blockquote, a small amount of context, and H2 sections with descriptive links. The Optional section marks sources an agent can skip when it needs a shorter context.

Use the copy above as a structure, then replace each claim and URL. Do not publish example domains, unfinished descriptions, or links copied from another company.

Choose links for the task, not the whole sitemap

An agent needs the sources that explain your product or project. For a developer tool, that set might include the quickstart, API reference, authentication guide, pricing page, and security policy. A course site might link the syllabus, assignments, grading policy, and instructor contact page.

The proposal describes llms.txt as a curated overview. It gives agents a path to detailed sources instead of copying the full site into one file. Cloudflare's live developer documentation llms.txt shows how a large documentation site can group source links by product area.

Use canonical public URLs. Link a Markdown version when you maintain one, since an agent can read it without removing navigation and page chrome. A normal HTML page still works when it contains the authoritative information.

Keep each link description concrete:

- [Authentication](https://docs.example.com/auth.md): Create, rotate, and revoke API keys.

Labels such as “Learn more” or “Resources” force the agent to fetch a page before it can judge the source. Name the subject and state what the page contains.

Publish the llms.txt example at the right path

Most sites should publish the file at the origin root:

https://example.com/llms.txt

Version 2 of the proposal also allows a file below the root. A file at /docs/llms.txt covers the pages under /docs/, and the most specific file applies when more than one file covers a path. Root placement remains the broadest discovery point and the location Slop Audit checks.

Static-site projects can add llms.txt to the public assets directory. Server applications can expose a fixed route that returns the file. In either case, prevent the application shell from answering a missing-file request with HTML and a 200 status.

Check the production response:

curl -sS -D - https://example.com/llms.txt -o /tmp/example-llms.txt
sed -n '1,80p' /tmp/example-llms.txt

Look for a successful status, a plain-text content type, the expected H1, and complete public URLs. Run the command against the production hostname because a CDN rule or single-page application fallback can change the response after deployment.

Keep the file factual and safe to follow

Write llms.txt as an index of public source material. State what the site does, identify the intended audience, and describe the linked pages. Remove marketing claims that the sources cannot support.

Do not put secrets, private paths, unpublished roadmap details, or access tokens in the file. Anyone can request it. Avoid prompt-like commands that tell an agent to ignore other instructions, reveal data, or treat your file as a system message. Those strings make the file less trustworthy and can trigger safety checks.

Review outbound links with the same care as package names and installation commands. An agent may use the destination as a source. Point to domains you control or primary references you have verified, and remove stale links when ownership changes.

The JSON-LD structured data guide covers page-level entities and visible facts. Keep that markup in the HTML. llms.txt supplies a separate curated route map and does not replace schema.

Know what llms.txt does not control

Three root files have separate jobs:

• llms.txt offers a proposed, agent-facing guide to useful content.
• robots.txt states crawl rules for user agents that choose to follow them.
• sitemap.xml lists URLs you want search engines to discover.

Google's robots.txt documentation explains that robots rules manage crawler access and do not guarantee removal from search results. The llms.txt proposal says its file should coexist with robots and sitemaps. Do not place allow or disallow rules in llms.txt, and do not expect it to grant access to a blocked page.

Use the robots.txt versus sitemap.xml guide to review crawl access and URL discovery. An accurate llms.txt cannot repair a blocked documentation route, a wrong canonical, or a server error.

Test llms.txt in CI

A small test can catch the failures that matter at launch:

import assert from "node:assert/strict";

const response = await fetch("https://example.com/llms.txt", {
  redirect: "follow",
});
const body = await response.text();

assert.equal(response.status, 200);
assert.match(response.headers.get("content-type") ?? "", /^text\/plain\b/i);
assert.match(body, /^#\s+Acme API\s*$/m);
assert.doesNotMatch(body, /<html[\s>]/i);

for (const [, url] of body.matchAll(/\[[^\]]+\]\((https:\/\/[^)]+)\)/g)) {
  const linked = await fetch(url, { method: "HEAD", redirect: "follow" });
  assert.ok(linked.ok, `${url} returned ${linked.status}`);
}

Some servers reject HEAD even though GET works. Use a bounded GET fallback if your linked services behave that way. Keep the link list short enough that the check respects rate limits and finishes within your CI budget.

Add the file to your post-deployment checks. Save the status, content type, H1, and link failures with the release identifier. The website regression testing checklist shows how to keep a reviewable baseline and separate a new failure from an accepted exception.

Check llms.txt with Slop Audit

Run Slop Audit against a public page on the production origin. The audit requests the root /llms.txt, checks for a usable text response, and looks for factual headings or links. It also flags common prompt-injection phrases and records a bounded sample as evidence.

Slop Audit checks the root file, not path-specific files such as /docs/llms.txt. It cannot prove that an AI provider will fetch the file, cite your pages, or improve search placement. Use the finding to verify deployment and content hygiene. Review the generated llms.txt fix draft before publishing it because you own the source choices and claims.

llms.txt release checklist

• Publish /llms.txt on the production origin with a 200 response and plain-text content type.
• Start with one H1 that names the site or project.
• Add a concise summary and grouped links to canonical public sources.
• Describe each linked page in words a developer can verify.
• Remove secrets, prompt-like commands, placeholders, and stale destinations.
• Keep robots rules, sitemap URLs, canonicals, and structured data in their own files.
• Test the deployed response and important links in CI.
• Run a public audit after release and record the evidence with the build.

Update the file when docs, pricing, policies, or product limits change. A short maintained guide gives an agent a better starting point than a long index nobody reviews.