Generate standard llms.txt files for AI search engines to help AI crawlers correctly understand and reference your website content
llms.txt is a website description file designed for AI large language models and generative search engines, similar to how robots.txt works for traditional search engines. Written in Markdown format and placed in the website root directory, it proactively informs AI crawlers about your brand's core information, priority pages, and detailed content paths. As AI search penetration continues to grow, configuring llms.txt has become an essential part of GEO (Generative Engine Optimization).
llms.txt uses Markdown format with the following structure: the first line is a title (H1), followed by a blockquote with a one-line description, then a detailed description paragraph. Links are organized into three tiers: Summary (core pages AI crawls first), Docs (detailed documentation), and Optional (supplementary information).
Fill in the website name and one-line description (required) — these are the first pieces of information AI reads. Add Summary links (core pages like homepage, product pages), then add Docs links (detailed documentation pages). Click "Copy" or "Download File" to get the llms.txt file, and upload it to your website root directory (same level as robots.txt). Visit https://yourdomain.com/llms.txt to confirm it's accessible.
Place llms.txt in the website root directory, at the same level as robots.txt. For example: https://www.example.com/llms.txt. Make sure the file is publicly accessible and not blocked by robots.txt for AI crawlers.
llms.txt itself is not a ranking factor, but it helps AI crawlers understand your brand and content more accurately and efficiently. When users ask questions related to your business on AI search engines, AI can reference your information more precisely, thereby increasing brand visibility and traffic. This is an important part of GEO optimization.
Currently, crawlers from major AI search engines like ChatGPT, Perplexity, and Claude all crawl text files in the website root directory. llms.txt as a community standard is rapidly gaining adoption, with more AI platforms starting to actively retrieve and reference information from this file.
Yes, they serve different purposes and both are needed. robots.txt controls crawler access permissions (what can and cannot be crawled), while llms.txt provides content guidance (how to understand the content after crawling). We recommend first configuring access rules with our Robots.txt Generator, then configuring content guidance with the llms.txt Generator.
llms.txt uses Markdown format because large language models have the strongest understanding of Markdown. Markdown's heading hierarchy, lists, links, and other structures can be accurately parsed by AI, making it more suitable as an AI-readable website description file than JSON or XML.