Article · AI Search10 min read · 12 August 2026

Does Your Irish Website Need llms.txt And Markdown Pages?

Understand the llms.txt proposal, optional Markdown alternatives, current platform limitations and when a controlled experiment is worth running.

Most Irish business websites do not need llms.txt or Markdown page copies as their first AI-search project. They need accessible canonical HTML, correct crawler/index controls, clear service content, internal links and trustworthy evidence. Once those foundations work, llms.txt can be a bounded experiment—not a ranking promise.

What the proposal actually says

The llms.txt proposal describes a Markdown file at /llms.txt—or within a subpath—that gives brief context and curated links to detailed resources. Its version-two proposal also describes optional Markdown representations and standard link relations connecting HTML, Markdown and the relevant llms.txt file.

The distinction matters: it is a proposal developed through community adoption, not an instruction that every major AI-search product promises to use. A valid file can still have no measurable discovery effect.

What it does not replace

Existing mechanismJobWhy llms.txt is different
robots.txtCommunicates permitted crawler pathsllms.txt is guidance, not access control
XML sitemapLists important canonical URLs for search discoveryllms.txt is curated context, not a complete index list
Canonical HTMLPrimary customer-facing contentMarkdown should not become an unmanaged competing truth
Structured dataLabels page entities/factsllms.txt points to resources rather than marking entity properties
Navigation/internal linksHelps people and crawlers discover relationshipsRemains a foundational site mechanism

Do not publish private information in llms.txt and assume a bot will respect its intent. Anything public at the URL should be treated as publicly retrievable.

What Google and OpenAI document

Google’s AI-features guidance says AI Overviews and AI Mode require ordinary Search index/snippet eligibility. It explicitly says no new machine-readable file, AI text file or special schema is required.

OpenAI’s ChatGPT Search guidance focuses on allowing OAI-SearchBot and published IP traffic for inclusion. It does not promise placement from an llms.txt file. OAI-SearchBot and GPTBot training controls are separate.

Therefore: a file may help a compatible agent navigate, but it does not fix blocked crawling, poor pages or missing evidence.

Use the priority ladder

Complete these layers in order:

  1. Important public pages return useful canonical HTML.
  2. Intended crawlers can access them through robots/CDN/WAF controls.
  3. Search indexation, canonicals, sitemaps and internal links are correct.
  4. Services, identity, evidence and authorship are clear and current.
  5. Structured data matches visible facts where appropriate.
  6. Only then test llms.txt or Markdown alternatives.

When an experiment may be sensible

The strongest use case is a content-heavy documentation or knowledge site where agents need a curated path into stable technical material. A business site may test it when it has:

  • substantial maintained guides, policies or technical references;
  • one owner for content and implementation;
  • automated generation from canonical sources;
  • server-log or referral measurement; and
  • no higher-priority indexing/content defects.

For a six-page brochure site, a new file may add more maintenance than value.

Implement it without creating a second website

Keep /llms.txt concise: business/project identity, important interpretation notes and curated links grouped by purpose. Link only to content that is public, accurate and useful.

If creating Markdown alternatives:

  • generate HTML and Markdown from one content source;
  • preserve meaning, dates, attribution and links;
  • use an appropriate alternate relation;
  • keep canonical governance explicit;
  • prevent Markdown routes entering the wrong sitemap/index flow;
  • test authentication and confidential-data boundaries; and
  • fail the build when versions drift.

Measure it as an experiment

Record the implementation date and intended hypothesis—for example, “compatible agents will fetch the guide links more reliably.” Monitor server logs for file and linked-resource requests, known AI referrals, cited URLs and factual accuracy in a fixed prompt benchmark.

Do not use a before/after mention count as proof when content, crawling or external authority changed simultaneously. Be prepared to remove the experiment if it becomes stale or produces no useful signal.

What to do next

Audit canonical pages and crawler access first. If those are sound and the site has a real knowledge corpus, prototype one small maintained llms.txt pointing to the strongest resources; defer Markdown mirrors until generation and drift controls exist.

Read the AI SEO Ireland guide for the complete priority model and ChatGPT visibility guide for testing. Hatch’s AI Search Optimisation service can run an AI crawlability review without treating an experimental file as a shortcut.

Common questions

Frequently Asked Questions

What is llms.txt?

It is a community proposal for a concise Markdown file that describes a site or path and links agents to selected useful resources, potentially including clean Markdown alternatives. It is not the same as robots.txt or an official universal web standard.

Does llms.txt improve AI search rankings?

There is no reliable basis for guaranteeing a ranking or citation improvement. It may help compatible agents navigate selected content, but selection depends on platform behaviour and the quality and accessibility of the underlying sources.

Does Google require llms.txt for AI Overviews?

No. Google explicitly says its AI features need no new machine-readable or AI text file, special schema or additional technical requirement beyond normal Search eligibility.

Is llms.txt the same as robots.txt?

No. robots.txt communicates crawler-access rules. The llms.txt proposal provides contextual guidance and links. Never treat llms.txt as an access-control or privacy mechanism.

Should every webpage have a Markdown copy?

No. Markdown alternatives add publishing and drift risk. Test them for content-heavy pages where clean machine-readable representation has a clear use case, and generate them from the same source as canonical HTML where possible.

Can Markdown pages create duplicate-content problems?

Public alternate URLs require deliberate canonical and linking governance. Keep content equivalent, identify the canonical HTML, avoid indexing conflicting stale versions and test how intended crawlers and search engines handle them.

Want Us To Look At Your Version?

Show us what is not working and we’ll give you a straight answer.

Book A Call
Ben Dunlea profile photo
Ben Dunlea

Founder of Hatch. Writing practical, jargon-free advice for businesses that want to get found and turn attention into enquiries.