Generative engine optimization (GEO), sometimes confused with answer engine optimization (AEO), is now an area of focus within search engine optimization (SEO). Organizations are wondering: how do we get chatbots like ChatGPT, Gemini, Claude, Perplexity and Google’s AI mode to mention my business or product?
Industry experts are proposing ways to help the chatbots understand website content.
One of those proposals is an llms.txt file, first recommended by Jeremy Howard in 2024.
As an evolution of the llms.txt file suggestion, Mintlify and Anthropic created the llms-full.txt option.
These file suggestions are based on the idea that structured markdown content may be easier for large language models (LLMs) to process. The value of markdown for LLMs is explained in this Webex post.
This post is specifically about the llms.txt file.
What is an llms.txt file?
With traditional SEO, sitemaps, index headers and robots.txt files are used to help search engines find a website’s pages.
The proposed llms.txt file offers to help large language models (LLMs) identify a website’s most important content.
An llms.txt file uses special formatting—called markdown—to distill information about a website. Markdown is believed to be useful for LLMs because it:
- Provides a simplified, text-only summary of a website’s main content
- Excludes the added code that can clutter a website’s page content (e.g., the menu links, images, JavaScript, tracking codes and ads that appear in the HTML version of a page)
The llms.txt file is placed in the root directory of a website. It acts as a ‘table of contents’ (so to speak).
What is an llms-full.txt file?
As noted above, the llms.txt file is meant to be an index, or a ‘table of contents’ with short descriptors of your website’s purpose and its pages.
The llms-full.txt file was later proposed as a version that contains the full text content of a website. This would have the advantage of excluding token-eating data such as navigation menus, javascript, ads and other code on web pages.
Instead, llms-full.txt can return more of the text-based content that the LLMs look for. It can also do so in one query, rather than multiple. Thus, it has the potential of saving tokens and reducing latency.
The llms-full.txt file is thought to be useful for deep content retrieval.
The llms.txt file is thought to be useful for quick summary retrieval.
Note: Jeremy Howard’s command-line application generates a llms-ctx-full.txt, which is similar to llms-full.txt. More on that here.
Not to be confused with .md files for individual URLs
The llms.txt and llms-full.txt markdown files should not be confused with individual page URL .md files. These additional .md files are also believed to be useful to some LLMs. It all depends on the context of the search. Like with llms.txt and llms-full.txt, the unverified value of .md files has induced skepticism.
Do AI search engines use the llms.txt or llms-full.txt files?
According to research at the time of writing, no AI company has officially stated that they use the llms.txt or llms-full.txt files to help deliver content to their users. The latest industry update on the topic seems to be around July, 2025.
However, early adopters are signaling their support by including llms.txt and llms-full.txt files on their websites. This includes brands like Cloudflare, Anthropic, Vercel and ElevenLabs, according to llms-text.com. Stripe, Zapier, NVIDIA and others are listed as adopters by getpublii.com.
This has led to some AI agents crawling or ‘signaling’ support for both the llms.txt and llms-full.txt files, as explained by Mintlify here (and in the aforementioned Publii article).
If AI tools don’t use llms.txt files, why should I create one?
Despite skepticism from some, there is anticipation by others that the llms.txt and llms-full.txt files will eventually be used by AI services. For this reason, some websites may want to create them as a future-proofing measure.
What is the proposed benefit of the llms.txt file?
It’s important to know that answer engines do not scan a website’s pages the same way that search engines do. AI tools only ‘reach’ small portions of it at a time. Those portions can be specific to a user’s “fan-out query.”
The hope is that llms.txt files will guide generative engines to pages with the most relevant, up-to-date information that could respond to a user’s query. For website owners, being cited efficiently by AI tools has the following benefits:
- Brands can be mentioned as solutions to a user’s problem (hopefully leading to a conversion)
- Links to the sourced content (within the answer) could generate website traffic
- Server resources can be conserved (since the llms.txt file may reduce token usage)
If your site is very large, or contains several non-essential pages, you would not want the LLMs getting ‘lost’ or misrepresenting your brand. This can happen if they:
- Navigate irrelevant or outdated content
- Ignore your most authoritative, ‘cornerstone’ pages
- Fail to find your core message due to limited memory capacity
For this reason, the llms.txt file, if adopted by AI companies, could help a bot ‘cut through the clutter’ and get straight to the content that’s useful.
What types of websites should use an llms.txt file?
The following scenarios are thought to be the best use-cases for an llms.txt file:
- Large, knowledge base websites (e.g. help centers with common FAQs, how-to’s, documentation, etc.)
- Blogs or news websites that get updated regularly
- E-commerce stores with thousands of products
- Educational or healthcare websites with several topics or departments
However, there could be more.
What should not go in an llms.txt file?
Unlike a sitemap, an llms.txt file is not a list of all your site’s pages.
It is also not a file meant to contain all the content within your site’s pages. For that, you would want to use the llms-full.txt format (see also: the llms-ctx-full.txt format).
Creators of an llms.txt file should aim for brevity, clarity and structure. Listing too much in an llms.txt file could render it unuseful; at some point, it would be the same as asking an AI tool to scan your entire site, which is counterintuitive to its purpose.
How much is ‘too much’ for an llms.txt file?
This could be relative, depending on the size of your website. If your business only offers three core services, there probably wouldn’t be harm in listing the three individual pages that talk about those services in your llms.txt file.
However, in the cases noted above, listing every knowledgebase article for a large software company may be too much.
That said, Vercel’s llms.txt file is over 47,000 words long. You can see what it’s like here.
Note that although it is long, it provides categorized, useful content for answer engines to navigate.
On a side note…
Vercel also famously said their llms-full.txt file is a “400,000 word novel.” (Our search found one on their site that is just over 179,000 words long, based on wordcounter.net).
How do I create an llms.txt file on my WordPress website?
There are multiple ways you can create an llms.txt or llms-full.txt file on your website.
Use a WordPress plugin
On a WordPress website, the easiest way to create an llms.txt or llms-full.txt file would be with a plugin that can automatically generate one for you.
A popular option for creating an llms.txt file is:
However, searching the WordPress repository for “llms.txt” can bring up additional results.
Currently, there are a few plugins that claim to also generate an llms-full.txt file. They include:
- All in One SEO (AIOSEO)
- Website LLMs.txt
- Auto LLMS (.txt) Generator
- LLMs.txt and LLMs-Full.txt Generator
At the time of writing, some of the above plugins have few reviews and installs.
Use an external generator
There are free and paid generators that can create the llms.txt file for you.
You may need to manually place the llms.txt file in your website’s root directory via FTP/SFTP using these methods.
Options include:
And more.
To also generate a llms-full.txt file, there are developer tools such as:
And more.
Note that an llms-full.txt file would need to be updated as your site’s content changes.
Ask AI to write an llms.txt file for you
Since we’re on the AI ‘train’, why not ask your favourite AI chatbot for help? 😉
If you ask an AI assistant to draft a starting version of an llms.txt file, be sure to review and edit it yourself.
Write your file manually
There may be cases where the automatic llms.txt generators are too simple for a website’s needs. For example, the Yoast plugin will create an llms.txt file with a limited number of recent pages on your website. However, the most recent pages are not necessarily the most important pages.
Plugins can also be limiting when it comes to custom post types in WordPress.
For more granular control over what goes in your llms.txt file, you can write it yourself.
Different tutorials suggest different ways of doing this. You may want to follow Jeremy Howard’s proposed recommendation here.
To conclude: llms.txt files have potential, but not promise (yet), to be AI search-friendly
At the time of writing, llms.txt and llms-full.txt files are widely talked about, but little utilized. However, the trend seems to be pushing in the direction of widespread adoption. This can be seen with large brands using them, some AI tools crawling them, and WordPress plugins offering to create them.
If you hope to future-proof your WordPress site by implementing the llms.txt model, you may want to start by testing the plugins that offer to do this for free.
For more advanced use-cases, paid services or developer tools may be necessary.
Image credit: Freepik.
Leave a Reply