--- title: How to make your website visible and digestible for LLMs description: >- AI search visibility is the new SEO. Whether ChatGPT, Claude, and Perplexity cite your site comes down to what you build around it. language: English url: >- https://www.voorhoede.nl/en/blog/how-to-make-your-website-visible-and-digestible-for-llms/ --- Blog # How to make your website visible and digestible for LLMs By [James](/en/team/james.md) 22 July 2026 ## Categories * [Web performance](/en/blog/tag/web-performance.md) * [Strategy](/en/blog/tag/strategy.md) - [How AI search actually works](#how-ai-search-actually-works) - [Make your site reachable and readable](#make-your-site-reachable-and-readable) - [Can the bots even reach you?](#can-the-bots-even-reach-you-) - [Add structured data (schema)](#add-structured-data-schema-) - [Serve markdown page alternatives](#serve-markdown-page-alternatives) - [Let users chat with your page](#let-users-chat-with-your-page) - [llms.txt: worth it?](#llms-txt-worth-it-) - [Make your content worth citing](#make-your-content-worth-citing) - [Write chunk-friendly content](#write-chunk-friendly-content) - [Answer the question first](#answer-the-question-first) - [Publish original data](#publish-original-data) - [Keep content fresh](#keep-content-fresh) - [Separate information from sales](#separate-information-from-sales) - [Earn off-site authority](#earn-off-site-authority) - [How to know it's working](#how-to-know-it-s-working) - [Wrapping up](#wrapping-up) This space moves fast. I've spent the last few years trying to keep up, reading the research and testing things on our own site to see what gets us cited. For websites, I've crystallised what works best to make them work with LLMs. That's what's below. More and more people ask LLMs their questions instead of typing them into Google, and most websites haven't caught up. SEO best practices still help, but LLMs have their own needs. Whether you build the site or write for it, there are concrete things you can do to make your content easier for them to find, read, and cite. Just know that most of this is what we know right now: it should stay relevant, but some of it won't be true in a few months, so always verify before you make changes. ## How AI search actually works When we ask new clients at De Voorhoede how they found us, the answer used to be their network or Google. Now we hear "Chat recommended you." So how does that work? An LLM's answer comes from one of two places: what the model already learned in training, or a live web search it runs in the moment for more up-to-date information. When it searches, it grabs a handful of pages from its prebuilt index, writes a single answer based on them, and cites the ones it used. That live-search path is the one you can influence, and it has a name: retrieval-augmented generation, or RAG. It powers every major AI search today. The big shift is the results page: there isn't one. No list of ten blue links to climb. The model reads a few sources, writes one answer, and either your page makes the cut or it doesn't. There's no position #2. So the goal changes from ranking number one to being one of the sources the model picks. Everything below is about that: making sure the model can reach your pages, read them cleanly, and trust you enough to cite you over someone else. Each model also has its own index. Google, OpenAI, and Anthropic crawl the web themselves and build separate ones, so what a given model can see, and how fresh it is, won't always match the Google results you're used to. ## Make your site reachable and readable First, the technical side. Can the model actually fetch your pages and read them without tripping over clutter? This part is mostly developer work. ## Can the bots even reach you? Before any of this matters, the AI crawlers have to be able to fetch your pages, and plenty of sites block them without knowing it. Do everything else right and you can still be invisible.\ \ There's two things you need to check. Start with your `robots.txt` and make sure nothing blocks the AI bots. Then check your CDN or firewall, because tools like Cloudflare can block AI bots at the edge, sometimes by default, no matter what `robots.txt` says. To confirm they're getting through, check your server logs to see the bots actually show up.\ \ It also helps to know there's no single "AI bot". There are three kinds: * **Training crawlers** (`GPTBot, ClaudeBot`) collect data for future models. * **Search crawlers** (`OAI-SearchBot, Claude-SearchBot`) build the index the model searches when it answers. This is the one that gets you cited. * **Live fetchers** (`ChatGPT-User, Claude-User`) grab a page when a user's prompt asks for it. They're controlled separately, so you can block the training crawler and still stay visible in AI search. Here's that in `robots.txt`: ``` # Block OpenAI's training crawler User-agent: GPTBot Disallow: / # But let its search crawler through User-agent: OAI-SearchBot Disallow: ``` ## Add structured data (schema) Structured data is markup that tells machines what your page is about without making them guess. It spells out the facts in a format built for parsing: this is an article, here's the author, here's the company, here's when it was published. The simplest way to start is JSON-LD, the format Google recommends and the easiest to drop in. Add the types that fit your content: `Article` or `BlogPosting` for posts, `Organization` for your company, `Person` for authors, `FAQPage` for Q\&A. Link your entities with `sameAs` so the model can connect your brand to its profiles and pin down who you are. The full vocabulary lives at [schema.org](https://schema.org/). It's proven for traditional search and gives models clean entity data to read. How much each model leans on it for citations hasn't really been measured, but it's cheap and standard, so it's worth adding. ## Serve markdown page alternatives Serve a clean markdown version of every page. It gives the model just the content: no nav, no footer, no unlabelled images, no markup to dig through. That means fewer tokens to process and a cleaner read, so the model spends its attention on what you actually wrote. We do this on our own site. Add `.md` to a page URL and you get the clean version, this post included. To make it discoverable, do three things: * Serve it with the right content type, `text/markdown; charset=utf-8`, so it renders as markdown instead of downloading as a file. * Add a `rel="alternate`" link in the page head pointing at the `.md`. * Add a visible link on the page so people and agents can find it. For the full setup, including content negotiation and headers, the [Website Specification](https://specification.website/spec/agent-readiness/markdown-source-endpoints/) goes deep. Anthropic, Stripe, Vercel, and Cloudflare all serve markdown versions of their docs. ## Let users chat with your page This one helps your readers rather than your visibility, but it's worth copying, and you've almost certainly seen it. Docs sites now ship a small "Copy page" menu: copy the page as markdown, open the raw markdown, or start a chat about it in ChatGPT or Claude. Claude's own docs and Vercel both have it. It builds straight on the markdown work above. The chat option opens an AI with a prompt prefilled to point at your page's markdown, so a reader can start asking questions in one click. Both ChatGPT and Claude take the prompt from the URL (`chatgpt.com/?q=...`). Why only those two? We'd happily add Mistral's Le Chat and Proton's Lumo, but neither supports this at the time of writing. We put this on all our blog posts, including the one you're reading right now. ## llms.txt: worth it? Probably not for citations, but it's cheap and some tools read it. `llms.txt` is a proposed file at your site root, a curated markdown map of your content, pitched as an "AI sitemap" like `sitemap.xml`. The evidence isn't kind. [SE Ranking checked 300,000 domains](https://seranking.com/blog/llms-txt/) and found no link between having an `llms.txt` and getting cited, only about 10% of sites use it, and Google has said it has no plans to support it. Where it does get used is developer tooling. It's a clean way to feed your docs to AI coding assistants. LangChain, for example, publishes an [llms.txt for its own docs](https://docs.langchain.com/llms.txt) and built a server to hand these files to IDE agents. So if you ship developer docs and your audience uses those tools, it's worth having. Add one if you already serve markdown, it's quick and harmless. Just don't count on it for visibility, or put it ahead of the things that actually work. ## Make your content worth citing Now the editorial side. Is your content actually worth citing? This part is mostly content work ## Write chunk-friendly content Retrieval (RAG) doesn't grab your whole page. It splits the page into chunks and pulls the few that match the question, so each chunk needs to make sense on its own. Write each section so it works without the rest of the page around it. Give it a clear heading, keep it focused on a single point, and answer that point inside the section instead of relying on something three paragraphs up. Short, self-contained sections work better than one long argument, because the model might only ever see one of them. That's also why the markdown version helps: clean structure makes for clean chunks. ## Answer the question first Put the answer in the first sentence, then explain it. The reader gets the point straight away, and so does the model. Phrase your headings as the questions people actually ask, and answer each one in the opening line. The model matches your content against a question, so a section that leads with the answer is easy to pull and cite. Everything after that first line is supporting detail: the why, the caveats, the example. This post does the same: the first line answers the heading, the rest backs it up. ## Publish original data Run your own research, surveys, or benchmarks and publish the numbers. When a model uses a fact, it has to attribute it, and if the fact came from you, you're the attribution. Original data is some of the most citable content you can make. Models lean on verifiable, sourced facts, and a number that only exists on your page leaves them no one else to credit. Repeat what everyone already says and you're just one of a dozen interchangeable sources. ## Keep content fresh Update your important pages and show when you last did it. AI search leans toward recent content, so a page that was clearly updated last month has an edge over one that looks five years old. Put a visible update date on the page and set `dateModified` in your schema. Revisit your cornerstone pages on a schedule, fix what's gone stale, and bump the date when you do. It's an uneven signal, stronger in ChatGPT and Perplexity, lighter in Claude. But it's one of the few you can change on demand, so it's worth the habit. ## Separate information from sales Keep your informational pages focused on answering, not selling. Models favour content that informs over content that pitches, so a page stuffed with sales copy and calls to action gets cited less than one that just answers the question. Put the conversion work somewhere else. Let your guides, docs, and explainers do the explaining, and keep the "book a demo" and "buy now" pages separate. You still get the sale, the model just has a clean page to cite on the way there. ## Earn off-site authority Some of what gets you cited doesn't live on your site at all. Models weigh how often you're mentioned across other trusted places, so being talked about elsewhere matters too, not just the work you do on your own pages. Earn mentions where it counts: industry sites, reviews, and active communities. Reddit carries real weight here, partly because [Google](https://blog.google/company-news/inside-google/company-announcements/expanded-reddit-partnership/) and OpenAI have paid [licensing deals](https://redditinc.com/news/reddit-and-google-expand-partnership) for its data, used both to train models and to surface Reddit in their answers. When reputable sources reference you, the model trusts you more and is more willing to cite your pages. Sometimes it cites the other site instead, and over time those mentions feed what the model knows about you, no search required. Keep in mind, this one is correlation, not a guarantee. Brands that get talked about tend to get cited, but one mention won't do it. Treat it as recognition you build over time, not a quick win. ## How to know it's working Start with your analytics. If people are arriving from `chatgpt.com`, `perplexity.ai`, and the like, AI tools are surfacing your pages and sending you traffic. It's the easiest signal, and you can check it today. The catch is that referral traffic only counts clicks. Much of AI use is zero-click: the model answers from your page and the reader never visits. So you can be cited often and still see small numbers. For the fuller picture, track your share of answers. Take a fixed set of questions your audience would ask, run them through the LLMs, and count how often you show up. Run each question a few times, since answers vary, and check each engine on its own. It's a chore by hand, so tools like [Promptwatch](https://promptwatch.com/) can run it for you. ## Wrapping up None of this is magic, and you don't need all of it. Most of it is good web hygiene aimed at a new goal: being the source a model reaches for. Make sure the bots can reach you, serve clean content, and be worth citing, and you've covered most of the ground. This groundwork pays off for your own site too. The same setup makes it easier to add features like AI-assisted search or a chat assistant: your content is already in clean chunks, and your `llms.txt` doubles as an editorial steer on which pages matter most. The specifics will keep shifting. New conventions like `llms.txt` get proposed, crawlers change their rules, and what helps in one model matters less in another. But the direction holds: clear, reachable, trustworthy content is what gets picked. Start with the basics and adjust as the tools do. ## Related blog posts * ### [Make your design system agent-ready](/en/blog/making-your-design-system-agent-ready.md) 30 Jun 2026 By Sjoerd * ### [How close to the user should your code be?](/en/blog/how-close-to-the-user-should-your-code-be.md) 30 Oct 2025 By Jasper * ### [Reclaiming Digital Sovereignty on European Infrastructure](/en/blog/reclaiming-digital-sovereignty-on-european-infrastructure.md) 29 Jan 2026 By Luuk [← All blog posts](/en/blog.md) [Return to top](#top)