Google’s llms.txt Guidance Splits Between Search and Lighthouse, Major LLMs Ignore It

Google has introduced an optional llms.txt audit in Lighthouse 13.3, a tool used by developers to test web pages for agentic browsing. The audit checks whether a site hosts an llms.txt file, which is intended to provide a machine‑readable map of a site’s content for large language models. However, the Search team has clarified that the file is not required for search visibility and is largely irrelevant to Google’s own search algorithms. John Mueller, a Google webmaster advocate, explained that markdown pages and llms.txt are primarily useful for developer documentation sites and are a temporary aid rather than a long‑term necessity.

The divergence in guidance has created confusion among marketers and developers. While Lighthouse treats llms.txt as an optional convention for browser‑based agents, Search explicitly states that the file is unnecessary for AI search and compares it to the deprecated meta‑keywords tag. Mueller also noted that major LLM providers—including OpenAI, Anthropic, and Google’s own Gemini—do not currently consume llms.txt files. Gemini 3.5’s AI Overviews cite only 38 % of pages that rank in Google’s top 10, and AI Mode citations overlap with traditional search rankings at a mere 13.7 %. These figures suggest that structured data and high‑quality content remain the primary signals for both Google Search and LLMs.

In practice, site owners should prioritize core SEO fundamentals—such as clear site architecture, structured data, and user experience—over investing in llms.txt or markdown pages unless they maintain developer documentation or are specifically testing agentic browsing with Lighthouse. Google has not announced a timeline for broader adoption of llms.txt across its products, and the Search team has not confirmed that any LLM provider currently uses the file. The split guidance underscores that agentic optimization is not a universal requirement for all sites.