How to Create an llms.txt File: Template & Guide (2026)
Learn how to create an llms.txt file with copy-paste templates. Master the manifest syntax, configure llms-full.txt, and optimize your site for AI search engines.
As generative AI search engines and developer tools (such as ChatGPT Search, Perplexity AI, Claude Artifacts, and Cursor) become primary discovery channels for millions of enterprise buyers and developers, web engineering teams face a crucial architectural question: how do you provide Large Language Models with fast, clean, token-efficient access to your site's documentation without forcing them to parse megabytes of raw HTML, execute client-side scripts, and strip advertising boilerplate?
Learning how to create llms.txt files provides the definitive answer. Similar to how robots.txt establishes crawling rules and sitemap.xml guides traditional search indexation, an llms.txt manifest establishes a curated, plain-text Markdown roadmap that AI agents can consume in a single inference call.
In this practical technical guide, you will master the creation and deployment of llms.txt: examine the formal specification, copy battle-tested production templates for SaaS and content platforms, configure companion llms-full.txt context dumps, automate build pipelines, and validate your manifest for Generative Engine Optimization (GEO).
The Anatomy of a Compliant llms.txt File
The llms.txt file format is governed by the llmstxt.org standard and built entirely on the CommonMark Markdown Specification. It is structured into five distinct syntactic components designed to be ingested directly into an LLM's system prompt or RAG retrieval engine:
+-------------------------------------------------------------------------+
| THE 5 SYNTACTIC COMPONENTS OF LLMS.TXT |
| |
| 1. H1 TITLE: |
| `# Project or Domain Name` |
| |
| 2. BLOCKQUOTE PROJECT SUMMARY (Mandatory for System Prompt Injection): |
| `> Short 1-2 sentence core value proposition and architecture.` |
| |
| 3. EXTENDED DESCRIPTION (Optional Context Paragraph): |
| Detailed paragraph describing capabilities and intended use. |
| |
| 4. H2 SECTION CATEGORIES: |
| `## Documentation Category` |
| |
| 5. BULLETED HYPERLINKS (Anchor + Absolute URL + 1-Sentence Summary): |
| `- [Document Title](https://example.com/url): Concise description.` |
+-------------------------------------------------------------------------+Strict Formatting Rules:
- Absolute Canonical URLs: Always use full HTTPS URLs (
https://example.com/docs/guide), never relative paths (/docs/guide). - One-Sentence Descriptions: Every link must be followed by a colon and a high-information-density description explaining the content of the target document.
- UTF-8 Encoding: Author the file in clean, unminified UTF-8 plain text.
Copy-Paste Production llms.txt Templates
Here are three validated templates designed for different business models:
Template 1: SaaS / Developer Platform
# BugViso
> BugViso is an automated website intelligence platform that audits Core Web Vitals under simulated 3G mobile networks, technical SEO, and AI Search Readiness (GEO).
BugViso combines a headless Chromium crawler with automated diagnostic engines to evaluate web speed, link integrity, and Generative Engine Optimization.
## Core Platform
- [Core Web Vitals Engine](https://bugviso.com/features#vitals): Measures LCP, INP, CLS, and TTFB under simulated Slow/Fast 3G networks.
- [AI Search Readiness (GEO)](https://bugviso.com/features#geo): Evaluates robots.txt AI permissions, /llms.txt structure, and content extractability.
- [Technical SEO Audit](https://bugviso.com/features#seo): Audits canonical tags, link graphs, duplicate SimHash content, and XML sitemaps.
## Engineering Guides
- [Cumulative Layout Shift Guide](https://bugviso.com/blog/how-to-fix-cumulative-layout-shift-cls): Deep-dive guide on eliminating visual jumps with explicit image sizing.
- [Interaction to Next Paint Diagnostics](https://bugviso.com/blog/what-is-inp-and-how-to-fix-it): Guide on eliminating main-thread long tasks and optimizing JavaScript execution.
- [Time to First Byte Optimization](https://bugviso.com/blog/how-to-reduce-ttfb-time-to-first-byte): Backend caching, CDN configurations, and database connection pooling.
## Optional
- [Complete Documentation Dump](https://bugviso.com/llms-full.txt): Full concatenated documentation context for comprehensive LLM ingestion.Template 2: Technical Blog / Content Publisher
# TechArchitecture Blog
> In-depth engineering tutorials, distributed systems architecture, and web performance optimization.
## Pillar Engineering Guides
- [Distributed Caching Patterns](https://example.com/blog/distributed-caching-patterns): Architectural analysis of Redis clustering, cache-aside, and write-through patterns.
- [Database Connection Pooling](https://example.com/blog/database-connection-pooling): Optimizing PgBouncer and connection pooling in serverless environments.
- [Web Performance Optimization](https://example.com/blog/page-speed-optimization-checklist-2026): Top-to-bottom speed optimization checklist for engineering teams.
## System Design Case Studies
- [Scaling to 100k Requests Per Second](https://example.com/cases/scaling-100k-rps): Infrastructure post-mortem on scaling microservices under extreme load.Template 3: Open-Source Software / Library
# FastSchema Validator
> Lightweight, zero-dependency JSON and TypeScript runtime schema validation library.
## Quickstart & API
- [Installation & Setup](https://example.com/docs/installation): Package manager installation (npm, pnpm, yarn) and environment requirements.
- [Core Validation API](https://example.com/docs/core-api): Complete API reference for string, number, object, and array validators.
- [TypeScript Type Inference](https://example.com/docs/type-inference): Generating static TypeScript types directly from runtime schemas.
## Optional
- [Full API Reference](https://example.com/llms-full.txt): Single-page complete documentation dump for developer IDE context injection.Creating the Companion llms-full.txt File
While /llms.txt acts as an index (Table of Contents), /llms-full.txt contains the complete unabridged text of all your primary documentation concatenated into a single Markdown file.
+-------------------------------------------------------------------------+
| THE STRUCTURE OF AN LLMS-FULL.TXT FILE |
| |
| # Project Name Full Documentation Dump |
| |
| --- |
| # Document 1: Getting Started Guide |
| [Complete raw Markdown body of Document 1...] |
| |
| --- |
| # Document 2: Core API Reference |
| [Complete raw Markdown body of Document 2...] |
| |
| --- |
| # Document 3: Performance Best Practices |
| [Complete raw Markdown body of Document 3...] |
+-------------------------------------------------------------------------+Why Create llms-full.txt?
Modern AI models (such as Claude 3.5 Sonnet and GPT-4o) possess context windows exceeding 128,000 to 1,000,000 tokens. An AI coding assistant like Cursor or Copilot can ingest your entire /llms-full.txt file in a single prompt, allowing it to write flawless integration code without making dozens of individual HTTP requests.
Step-by-Step Implementation and Server Configuration
Follow this deployment workflow to host your manifest files in production:
+-------------------------------------------------------------------------+
| LLMS.TXT DEPLOYMENT WORKFLOW |
| |
| Step 1: Save `llms.txt` and `llms-full.txt` in your web root |
| (e.g., `/public/llms.txt`) |
| | |
| v |
| Step 2: Configure Web Server MIME Types & Headers |
| -> `Content-Type: text/plain; charset=utf-8` |
| -> `Access-Control-Allow-Origin: *` (Enables IDE fetch) |
| | |
| v |
| Step 3: Test Accessibility via cURL |
| `curl -I https://example.com/llms.txt` |
| | |
| v |
| Step 4: Verify Zero 404 Links or Redirect Chains Inside File |
+-------------------------------------------------------------------------+Nginx Server Configuration Example
# /etc/nginx/conf.d/llms.conf
location = /llms.txt {
default_type text/plain;
add_header Content-Type "text/plain; charset=utf-8";
add_header Access-Control-Allow-Origin "*";
add_header Cache-Control "public, max-age=86400";
}
location = /llms-full.txt {
default_type text/plain;
add_header Content-Type "text/plain; charset=utf-8";
add_header Access-Control-Allow-Origin "*";
add_header Cache-Control "public, max-age=86400";
}Automated llms.txt Generation in Modern Frameworks
Rather than maintaining your llms.txt file manually, automate its generation during your build process.
Next.js App Router Dynamic Generator
Create a dynamic route handler at app/llms.txt/route.ts:
// app/llms.txt/route.ts
import { getAllArticles } from "@/lib/content";
export async function GET() {
const articles = await getAllArticles();
let manifest = `# BugViso\n\n`;
manifest += `> Automated website performance, Core Web Vitals, and AI Search Readiness auditing.\n\n`;
manifest += `## Technical Documentation & Guides\n`;
for (const doc of articles) {
manifest += `- [${doc.title}](https://bugviso.com/blog/${doc.slug}): ${doc.summary}\n`;
}
manifest += `\n## Optional\n`;
manifest += `- [Full Documentation Context](https://bugviso.com/llms-full.txt): Complete unabridged documentation dump.\n`;
return new Response(manifest, {
status: 200,
headers: {
"Content-Type": "text/plain; charset=utf-8",
"Access-Control-Allow-Origin": "*",
"Cache-Control": "public, max-age=86400",
},
});
}Information Architecture for AI: Curating vs. Dumping URLs
A critical mistake made by engineering teams is converting an XML sitemap directly into an llms.txt file.
+-------------------------------------------------------------------------+
| XML SITEMAP VS LLMS.TXT SELECTION |
| |
| XML SITEMAP (`sitemap.xml`): |
| - Scope: Exhaustive (Contains every indexable URL, 50,000+ pages) |
| - Goal: Complete discovery for search engine indexers |
| |
| LLMS.TXT MANIFEST (`llms.txt`): |
| - Scope: Highly Curated (Top 20 to 80 authoritative documents) |
| - Goal: Clean, high-density context for LLM reasoning & RAG retrieval |
+-------------------------------------------------------------------------+The 20–80 Rule for AI Manifests
Select the top 20 to 80 core architectural guides, API references, and product definitions that represent your platform's authoritative knowledge base. Omit paginated category archives, ephemeral news items, and duplicate parameter pages.
For a deeper understanding of how AI search engines process curated manifests, review our guide on what is llms.txt? the AI website manifest guide.
How BugViso Audits and Validates Your llms.txt Implementation
Deploying an llms.txt file is only effective if its syntax complies with CommonMark standards and every referenced URL resolves with a clean HTTP 200 OK status.
+-------------------------------------------------------------------------+
| BUGVISO LLMS.TXT & AI MANIFEST AUDIT ENGINE |
| |
| [Target Domain Crawled via Headless Chromium] |
| | |
| v |
| [AI Readiness Diagnostic Pipeline (`ai_readiness.py`)] |
| | |
| +---> 1. Manifest Endpoint Discovery |
| | (Pings `/llms.txt` and `/llms-full.txt` at root) |
| | (Validates HTTP 200 OK & `text/plain` MIME type) |
| | |
| +---> 2. Markdown Syntax & Schema Validator |
| | (Verifies H1 title, blockquote, & H2 categories) |
| | (Ensures CommonMark compliance) |
| | |
| +---> 3. Token Weight & Context Window Profiler |
| | (Measures byte size & estimated token count) |
| | (Flags oversized manifests that risk truncation) |
| | |
| +---> 4. Concurrent Link Health Integrity Checker |
| | (Tests every URL declared in `llms.txt`) |
| | (Flags 404 broken links & redirect hops) |
| | |
| v |
| [0-100 AGGREGATE GEO CITABILITY SCORE + ACTIONABLE PLAYBOOK] |
+-------------------------------------------------------------------------+When you run an automated website scan with BugViso, the AI Readiness module evaluates your llms.txt implementation across five validation layers:
- Endpoint & MIME Type Discovery:
BugViso confirms the presence of
/llms.txtand/llms-full.txtat your domain root, ensuring appropriatetext/plain; charset=utf-8headers and fast TTFB response times. - Syntax & Structure Parsing: The engine validates CommonMark compliance, verifying the presence of an H1 title, blockquote project summary, category groupings, and properly formatted link descriptions.
- Token Weight Profiling: BugViso calculates total byte weight and estimated token counts to guarantee your manifest can be ingested without context truncation.
- Concurrent Link Integrity Testing:
Every URL declared in your
llms.txtfile is crawled concurrently to ensure zero 404 broken links, 5xx server errors, or multi-hop redirect chains exist in your manifest. - 0–100 GEO Citability Score: Findings are synthesized into an overall 0–100 AI Search Readiness Score with prioritized developer remediation actions delivered in both the interactive dashboard and downloadable executive PDF report.
The checks behind this are covered on the AI search readiness checker page.
Common Mistakes When Creating llms.txt Files
Avoid these frequent mistakes when authoring and deploying your manifest:
| Common Mistake | Consequence |
|---|---|
| Using Relative Link Paths | AI agents cannot resolve targets |
| Omission of Blockquote | LLM lacks high-level system context |
| Broken Links in File | RAG retrieval fails on target fetch |
| Misplaced File Location | Bots fail to discover /docs/llms.txt |
1. Using Relative URLs
AI agents ingesting an llms.txt file do not always infer the origin hostname. Writing "- [Guide](/blog/guide)" forces the agent to guess the root domain. Always declare full absolute canonical URLs: "- [Guide](https://example.com/blog/guide)".
2. Leaving Broken Links in the Manifest
If an AI search engine follows a URL listed in your llms.txt that returns a 404 error or a 3-hop redirect chain, the retrieval process fails, and the model skips your domain. To maintain clean redirect architectures, consult our guide on how to fix redirect chains and loops.
Frequently Asked Questions About Creating llms.txt
Do I need both llms.txt and llms-full.txt?
While /llms.txt is the primary standard required for AI search engines, providing /llms-full.txt is highly recommended if you serve developers, technical users, or developer IDEs (like Cursor) that ingest complete documentation in a single context window.
Can I include query parameters in llms.txt URLs?
Avoid tracking parameters (like ?utm_source= or ?ref=) in your llms.txt URLs. Use clean, canonical URLs to ensure AI agents retrieve the authoritative version of each document.
Where should I host llms.txt on a multi-subdomain website?
Each subdomain should host its own llms.txt file (e.g., https://docs.example.com/llms.txt and https://example.com/llms.txt) to accurately reflect the content available on that host.
How do AI agents discover my llms.txt file?
AI search crawlers and developer agents automatically probe https://example.com/llms.txt when initiating knowledge extraction on a domain. You can also reference it in your robots.txt comments or link to it in your website footer.
Does llms.txt replace traditional Schema.org structured data?
No. Schema.org JSON-LD structured data and llms.txt serve complementary roles. JSON-LD establishes machine-readable entities in HTML markup, while llms.txt provides plain-text Markdown content for Large Language Model context ingestion.
Summary and Action Plan
Creating an llms.txt file is one of the most effective optimizations for Generative Engine Optimization (GEO): host a CommonMark Markdown file at /llms.txt, provide a concise blockquote project summary, curate 20 to 80 high-value canonical URLs with one-sentence descriptions, serve clean text/plain headers, and optionally deploy an /llms-full.txt documentation dump.
To validate your llms.txt file syntax, verify that all declared URLs return clean 200 OK responses, and receive a comprehensive 0–100 AI search readiness benchmark, running an automated BugViso AI readiness scan verifies your llms.txt syntax, tests link integrity, and scores your GEO citability.
See where your site stands
Run a free BugViso audit for SEO, speed, accessibility and AI search readiness — with fixes you can ship today.