How to check if your site has a valid llms.txt file
A practical guide to checking and fixing your llms.txt file so AI crawlers can find and cite your content correctly.
Your site is probably invisible to AI crawlers right now
If you've searched for an llms.txt checker, you're probably already suspicious that something's off. The file is new enough that most guides skip the practical part: how do you actually verify your llms.txt exists, parses correctly, and gives AI systems the context they need? AuditZap's llms.txt file check (in the AI Visibility category) surfaces exactly this, and in the last 30 days it's flagged the issue on 53 out of 136 completed audits. That's a lot of sites inadvertently opting out of AI-driven traffic.
Why llms.txt matters for AI visibility
The llms.txt convention (proposed by Answer.AI and gaining traction across major AI crawlers) gives large language models a structured summary of what your site is, what pages matter, and how your content should be used. Without it, an AI crawler has to infer all of that from your HTML, which means it might cite a blog index page instead of a cornerstone article, or skip your site entirely in favour of a competitor who made the signal explicit.
This isn't about rankings in the traditional sense. It's about whether Perplexity, ChatGPT browsing, Claude, and similar tools surface your content accurately when someone asks a question your site answers. As AI-generated answers replace a growing share of zero-click searches, your llms.txt is part of the new citation stack. Ignoring it is the same move as ignoring sitemaps in 2008.
AuditZap pairs the llms.txt check with related AI Visibility checks like answer engine schema and question-format headings, because all three work together to make your content AI-readable.
How to check and fix your llms.txt
Step 1: Confirm the file exists at the right location.
The file must live at the root of your domain:
https://yourdomain.com/llms.txt
Open that URL in a browser. If you get a 404, the file isn't there. If you get a redirect, check the chain: AuditZap's Redirect chains check will flag anything routing through more than one hop, which matters here because some crawlers won't follow redirects for this file.
Step 2: Validate the format.
A valid llms.txt file follows a simple markdown-like structure:
# Your Site Name
> One-sentence description of what your site is about.
## Key pages
- [Page title](https://yourdomain.com/page): What this page covers.
- [Another page](https://yourdomain.com/other): What this covers.
## Excluded
- [Admin](https://yourdomain.com/admin): Do not index.
The most common errors are: missing the top-level # heading, broken relative URLs (use absolute URLs), and files served with the wrong content-type header. Serve it as text/plain or text/markdown.
Step 3: Cross-check your robots.txt.
AuditZap's Robots.txt blocking pages check regularly catches sites that have blocked /*.txt patterns or the root path in robots.txt. If your robots.txt disallows crawlers from reading llms.txt, the file is useless. Check that your robots.txt does not block the file path, and ideally add an explicit allow:
User-agent: *
Allow: /llms.txt
Step 4: Check for a companion llms-full.txt.
Some implementations include an llms-full.txt with expanded content for crawlers that want more detail. This isn't required, but it's worth noting in your llms.txt:
## Extended content
- [Full context](https://yourdomain.com/llms-full.txt)
Step 5: Audit the pages you're pointing to.
The links inside llms.txt are only as good as the pages they reference. Run those URLs through AuditZap to confirm they don't return 4xx errors (caught by the Broken internal links check) and that each one has a valid title tag, meta description, and canonical tag. Pointing an AI crawler at a page with a missing H1 or broken canonical undermines the whole exercise.
Check it before the next AI crawler visits
An llms.txt file takes about 20 minutes to create correctly. The cost of not having one is harder to measure but straightforward to reason about: AI systems will describe your site using whatever signals they can find, which may not be the signals you'd choose.
If you want to see whether your llms.txt passes alongside the rest of your technical stack, run an AuditZap audit. The llms.txt file check sits alongside the broader AI Visibility category so you can see how AI-readiness connects to your structured data, heading hierarchy, and answer engine schema in one view.