AI bots are crawling the web at full speed right now. Some are indexing content, some are training models, and others are answering questions without ever sending users to your website. Naturally, SEOs and developers are asking an important question:
Should we make websites easier for AI to read?
That’s where Markdown enters the conversation.
Recently, there’s been a wave of discussion around serving Markdown versions of webpages specifically for AI bots. Some people think it’s the future of AI visibility. Others think it’s completely unnecessary. Google’s John Mueller even called it “a stupid idea.” Which is about as subtle as a brick through a window.
So, what’s actually going on here?
First of all… what is Markdown?
Markdown is a lightweight way of formatting text.
Instead of using complicated HTML tags, Markdown keeps things simple:
# Heading
## Subheading
**Bold text**
- Bullet points
If you’ve ever written in Slack, Notion, GitHub or Reddit, you’ve probably used Markdown without realising it. Think of it as the “plain English” version of a webpage. No fancy styling. No JavaScript fireworks. Just the content and basic structure.
For humans, HTML creates the visual website experience.
For machines, Markdown can sometimes be easier and cheaper to process.
And yes, cheaper matters, especially when AI companies are chewing through billions of webpages like a dog left alone with an open fridge.
Why are people talking about Markdown now?
The recent buzz largely came from Cloudflare introducing a feature called “Markdown for Agents” to request web pages in Markdown format via HTTP content negotiation headers.
In simple terms, Cloudflare can automatically convert webpages into Markdown when an AI bot requests them. Agents like Claude Code already send these accept headers with their requests. This is a compelling infrastructure-level story showing how the whole web stack is adapting.
Why? Because HTML is surprisingly bloated.
A normal webpage contains navigation, menus, scripts, styling, popups, cookie banners, tracking code, and enough clutter to make Marie Kondo weep. AI systems don’t need all of that. They mostly want the actual content.
Cloudflare showed an example where a page dropped from over 16,000 tokens in HTML to around 3,000 tokens in Markdown. That’s an 80% reduction. Since AI companies pay for processing power and token usage, leaner content could theoretically make websites more attractive for AI systems to crawl and use.
Does Markdown help AI understand content better?
This is where things get messy.
The short answer is: Maybe slightly, but probably not in the magical way people hope.
LLMs are already very good at understanding HTML. They’ve been trained on enormous amounts of web content for years. Google’s John Mueller pointed this out directly when discussing the idea of building Markdown-only versions of pages for LLMs.
There’s also concern that stripping pages down to Markdown removes useful context.
Technical SEO consultant Jono Alderson argues that webpages are more than just blocks of text. Layout, navigation, relationships between elements and overall structure all provide meaning.
A recipe page, for example, isn’t just ingredients and instructions. The surrounding context helps machines understand what type of content it is, how important elements relate to each other, and how users interact with it.
Flattening everything into plain text may actually remove signals rather than improve them.
But AI models do love Markdown…
Here’s the confusing bit.
Researchers have validated the use of Markdown for system prompts, presenting the technique as advantageous in terms of clarity, organisation, and the model’s ability to follow multi-step instructions with complex examples. The use of Markdown improves readability, syntactic consistency, and compatibility between models.
That means LLMs are naturally very comfortable reading Markdown-formatted text. And this has led some people to assume:
“If AI understands Markdown well, then websites should serve Markdown.”
However, reading and understanding something well isn’t the same as needing it. Humans can read handwritten notes on napkins too. That doesn’t mean restaurants should replace menus with biro scribbles.
Then, is markdown actually useful for SEO?
Right now, there’s very little evidence that Markdown improves AI visibility or citations.
Research and discussions so far suggest:
- AI bots can already process HTML perfectly well
- Google does not recommend Markdown-only pages
- There’s no proven ranking or citation benefit
- Structured data and clean HTML still matter far more
That said, Markdown may still become useful operationally.
For example, reducing crawl costs, helping AI agents process pages faster, improving efficiency for tools that specifically request Markdown, or simplifying content extraction.
So this may end up being more of an infrastructure optimisation than an SEO breakthrough.
The bigger issue nobody talks about
The real story here probably isn’t Markdown itself. It’s the growing battle between websites and AI crawlers.
AI bots are consuming huge amounts of content across the web, often without sending meaningful traffic back. Publishers are increasingly looking for ways to control, optimise or monetise AI access. Markdown is simply one piece of that bigger shift.
The more interesting question isn’t “should I add a Markdown version of my site?” It’s “how do I stay useful and visible in a world where AI is the first stop for most searches?” That’s a conversation about content quality, structured data, brand authority and genuine expertise, not file formats.
Final thoughts
Do we recommend the use of Markdown files for AI and SEO?
Markdown files aren’t useless for AI. But they’re probably not the revolutionary SEO tactic some people want them to be either.
AI systems already understand webpages remarkably well. Cleaning up bloated websites, improving accessibility, using structured data properly and making content genuinely useful are still the smarter long-term strategies.
Markdown may help AI process content more efficiently in some cases. But turning your entire website into robot-friendly plain text just because AI bots exist feels a bit like removing all the seats from your car to improve fuel economy.
Technically clever. Slightly concerning. And probably not what most people actually need.
Not sure where your website stands with AI?
Our Innovation Team keeps a close eye on visibility and presence, separating the genuine opportunities from the noise. If you want a straight answer on what’s actually worth your time, get in touch with us and let’s talk.
