You’ve probably seen big promises lately: “Install this plugin and you’ll rank in AI!” or “Add llms.txt / ai.txt and ChatGPT will feature your site!”
It sounds exciting, especially if you’re worried that AI answers are replacing normal search results. But right now, a lot of this is more hype than reality.
There Is No Single “AI Ranking”
When people talk about “ranking in AI,” they usually make it sound like there’s one big list, like Google’s search results.
In reality, every AI tool (ChatGPT, Perplexity, Gemini, etc.) works differently. Some:
- Use their own training data
- Use search engines in the background
- Pull from special databases or partner APIs
There is no single, shared “AI ranking system” you can optimise for with one file or one plugin.
What robots.txt Actually Does
Before we talk about llms.txt and ai.txt, it’s important to understand robots.txt. This is a small text file on your site that tells well-behaved bots what they’re allowed to crawl.
Search engines like Google and Bing look at robots.txt. Many AI companies say their crawlers do too. With it, you can:
- Allow or block certain bots from reading your content
- Protect private or sensitive areas of your site
What robots.txt does not do:
- It does not “boost your rankings” in search or in AI
- It does not guarantee that every bot will behave (some ignore it)
Think of robots.txt as a basic access control, not a magic SEO or “AI ranking” switch.
What About llms.txt and ai.txt?
You may have heard about llms.txt or ai.txt as “the new standard for AI SEO.” Here’s the simple truth:
- They are experimental ideas, not widely adopted standards
- Most major AI systems have not said they use them for anything important
- There’s no evidence they improve how often AI tools show or “rank” your site
If you want to add them as an extra way to express your preferences, that’s fine. But they should be treated as nice-to-have experiments, not must-have ranking tools.
So How Do AI Tools Actually Use Your Site?
Most AI tools interact with your content in a few basic ways:
- Some crawlers collect public pages to help train models
- Some tools search the web in real time and summarise the pages they find
- Some use official feeds, APIs, or trusted databases
In all of these cases, the same boring things still matter:
- Your site needs to be easy to crawl (not blocked by robots.txt)
- Your content needs to be clear, helpful, and well-structured
- Your brand needs to show up in normal search and trusted sources
There is no special “AI-only trick” that replaces this.
How Not to Get Fooled
Here are some red flags to watch out for:
- “We’ll make you rank in AI with one file.” There is no single AI ranking system, and no single magic file.
- “Install our plugin and ChatGPT will feature you.” ChatGPT doesn’t use random WordPress plugins as a ranking signal.
- “llms.txt / ai.txt is the new SEO standard.” Not yet. Maybe in the future, but right now it’s mostly marketing language.
If someone can’t clearly explain which AI tool you’ll show up in, how it works, and how they’ll measure it, treat the offer with caution.
What Actually Helps (Today)
- Use
robots.txtto clearly tell crawlers what they can and can’t access - Keep doing solid, normal SEO: fast site, good content, clear structure
- Make sure your brand is present on trusted sites and data sources in your space
- Review who is crawling your site and update your rules when needed
AI will keep changing, and standards like llms.txt or ai.txt might become more useful over time. But for now, the safest approach is simple: ignore the hype, control what you can with robots.txt, and focus on content and visibility that help both search and AI systems.