# Why Knowledge Freshness Is AI Infrastructure

> Published by **Knowledge Company** (knowledgecompany.ai).
> Canonical source: https://knowledgecompany.ai/blog/knowledge-freshness-ai-infrastructure
> Markdown mirror, generated 2026-09-02. Cite the canonical URL.

Peer-reviewed evidence that one outdated document degrades AI answers far more than irrelevant ones, why models cannot filter staleness themselves, and how Azure, Vertex, Glean and Google's Open Knowledge Format now treat freshness as infrastructure.

Knowledge Infrastructure

What the research actually shows about stale documents in AI systems — and why freshness is now being built into the retrieval layer rather than left to whoever last edited the page.

Published Jul 22, 2026

Updated Sep 1, 2026

Verified Sep 1, 2026

12 min read

In short

Knowledge Company's review of the peer-reviewed evidence finds that knowledge freshness has moved from content hygiene into the AI infrastructure layer. Peer-reviewed work (HoH, ACL 2025) shows an outdated document degrades answers even when the correct current document is retrieved alongside it — and that this is specific to staleness rather than general retrieval noise: six irrelevant passages moved a model's composite score by under two points, while one outdated passage cost more than twenty-four. Models cannot filter it themselves; on ClashEval the best model tested adopted incorrect retrieved content 15.7% of the time despite holding the correct answer, and its protective instinct keys on the model's own prior knowledge, so it largely disappears for proprietary internal documents. Scale does not help either: on FreshQA no model without retrieval exceeded 15% on fast-changing questions, and enlarging T5 from 770M to 11B parameters tripled never-changing accuracy while leaving fast-changing accuracy flat. Meanwhile the infrastructure layer is absorbing the problem — Azure AI Search ships freshness-aware retrieval with an ISO 8601 boosting duration, Vertex AI Search recrawls on a best-effort basis tied to query volume, Glean records who verified a document and when, and Google Cloud's Open Knowledge Format defines verified, status and stale_after fields, where a concept is stale once the current time passes stale_after. The limit of this evidence is that every benchmark cited enriches conflict deliberately, so the literature establishes severity when stale content is present and says nothing about how often it is present in a real knowledge base.

## The evidence for this article was this article

Until 1 September 2026, the page you are reading was 92 words long and carried a label reading “5 min read.” At a normal reading pace those 92 words take about twenty-four seconds. The label overstated the article by roughly twelve times, on a post about knowledge going stale, published by a company that sells the detection of knowledge going stale.

It got worse on inspection. The post displayed three dates — Published, Updated and Verified — and all three were identical. “Verified” therefore asserted nothing beyond “published.” The one mechanic we argue for most insistently, that verification is a separate act with its own timestamp, was not running on our own blog. And none of the three dates existed anywhere in the page markup as *datePublished* or *dateModified*, so no machine reading the page could tell how fresh it was even if it wanted to.

We found this by auditing our own corpus, which is the only corpus we have unlimited permission to audit. The reading-time labels were recomputed from the actual word count, the verification dates were separated from the publication dates, and this article was rewritten into the thing it had been claiming to be. That is the whole argument in miniature: a small claim nobody was looking at, wrong for six weeks, in the place where it was most embarrassing to be wrong.

What follows is the case that this class of problem is no longer a content-hygiene concern. It is an infrastructure concern, and the research on retrieval-augmented systems is now specific enough to say why.

## A stale document is not the same thing as noise

The strongest available evidence comes from [HoH, published at ACL 2025](https://aclanthology.org/2025.acl-long.301/) by Ouyang and colleagues. The authors built 96,124 question-answer pairs over 219,463 documents by diffing Wikipedia snapshots against each other, which gives them something rare: pairs of documents that are genuinely the same claim at two points in time, one current and one superseded.

The headline result is that an outdated document degrades the answer *even when the correct current document is also in the retrieved set*. In the condition where the current passage is present and one outdated passage is added alongside it, Llama-3.1-70B falls from 89.12 to 72.95 on the paper's composite score, while its rate of actively misleading answers rises from 4.12% to 10.05%. Llama-3.1-8B falls from 85.70 to 50.93. Qwen-2.5-7B falls from 76.31 to 33.93, with misleading answers rising from 3.51% to 21.68%. GPT-4o, checked on a 1,000-sample subset, goes from 89.5 to 65.5.

Those numbers are worth reading carefully. They are percentage points on a composite score that awards +1 for a correct answer, 0 for a missing one and −1 for a misleading one, graded by a model judge the authors validated against human annotators. “Harmful,” in their scoring vocabulary, means incorrect or misleading — not unsafe. And the corpus is Wikipedia served by a mock search engine, which the authors themselves note is far tidier than real-world outdated information.

But the result that matters most for anyone running a knowledge base is not the headline. It is the control.

### The control is the finding

The same paper measured what happens when you add six *irrelevant* passages to the context instead of one outdated one. Six irrelevant passages moved Llama-3.1-70B's accuracy and misleading-answer rate by under one percentage point, and its composite score by under two. One outdated passage cost it more than twenty-four.

This is the sentence to take away from the entire literature. Retrieval noise is close to free. Contradictory stale content is not. A document that is merely unrelated to the question gets ignored; a document that answers the question with last year's answer gets believed. They are different failure modes with different fixes, and the common instinct — “our retrieval just needs to be more precise” — addresses the cheap one.

## The model will not catch it for you

The obvious reply is that a sufficiently good model should notice the contradiction and prefer the newer source. [ClashEval](https://arxiv.org/abs/2404.10198), from Wu, Wu and Zou at Stanford and presented at NeurIPS 2024, tested that directly across 1,200-plus questions and six domains. It measures how often a model abandons its own correct prior belief in favour of incorrect retrieved content.

Claude 3 Opus did this 15.7% of the time, Claude 3 Sonnet 20.1%, Gemini 1.5 24.5%, Llama-3 26.4%, GPT-4o 30.4%, GPT-3.5 31.3%. The best model tested adopted wrong context roughly one time in six when it already knew better. The authors' own summary of the task is that discerning “when it is wrong in light of correct retrieved content” and rejecting “cases when the provided content is incorrect” remains difficult. Their proposed mitigations are external — arbitrating with token probabilities lifts GPT-4o from 61.5% to 69.3%, and a calibrated version reaches 75.4% — which is itself the point. The fix sits in the wrapper, not in the model's judgement.

There is one protective effect, and understanding its limit is important. Models resist retrieved content more strongly the more implausible it is; the researchers graded this with numeric perturbations from 0.1x to 10x and date shifts of twenty years. But that instinct keys on deviation from what the model already believes. For an internal pricing table, a security control, a support policy or a deprecated internal process, the model has no prior at all. There is nothing for the implausibility check to fire against, so the protection largely disappears exactly where enterprise retrieval operates.

A [2026 preprint](https://arxiv.org/abs/2605.11574) testing five current-generation models — Claude Sonnet 4.6, GPT-5.5, Gemini 2.5 Flash, Llama 4 Maverick and DeepSeek V3, across 9,970 API calls — adds a detail that should worry anyone who has written a retrieval prompt. Task framing alone flips context-following from near 100%, when the task is posed as one the supplied document is meant to answer, to somewhere between 6% and 71% when the same question is posed as general knowledge. How you word the instruction changes your exposure to a bad chunk more than the chunk does, and production retrieval prompts are almost always worded the first way.

One honest caveat travels with ClashEval. Its perturbations are fabricated errors, not genuinely stale content that used to be true, and the authors explicitly warn that their dataset has an enriched error rate and that the figures are “not meant to represent bias rates in the wild.” The mechanism transfers; the rates do not.

### Staleness is a named category, with a specified correct behaviour

This is not a framing we invented. [DRAGged into Conflicts](https://arxiv.org/abs/2506.08500), from Google Research and Bar-Ilan University, builds a five-category taxonomy of retrieval conflicts — no conflict, complementary information, conflicting opinions, conflict due to outdated information, and conflict due to misinformation — and gives the fourth the shorthand “Freshness.” It accounts for 62 of the 458 annotated cases. Their canonical example is a question about how many countries recognise same-sex marriage, which returns 37, 38 or 35 depending purely on when the retrieved passage was written.

The specification of correct behaviour is explicit: the system should “prioritize the up-to-date information, while optionally acknowledging the presence of outdated sources.” Meanwhile the [EMNLP 2024 survey of knowledge conflicts](https://aclanthology.org/2024.emnlp-main.486/) makes inter-context conflict one of three top-level types and lists exactly two causes beneath it: misinformation, and outdated information. Staleness is one of two named causes, not an implied one.

Do not read that 62-of-458 as a prevalence rate. The queries were deliberately drawn from datasets known to contain conflicts. It is 13.5% of a conflict-enriched benchmark, and nothing more.

## You cannot buy your way out with a bigger model

The parametric side of this is worse than the marketing suggests. [Set the Clock](https://aclanthology.org/2024.findings-acl.938/), from the University of Washington and AI2, found that base pretrained models answer time-sensitive questions using knowledge several years older than their stated cutoff. LLaMA2-70B, with a 2022 cutoff, peaks at 2019. LLaMA1-65B peaks at 2018. GPT-3 peaks at 2017. Every base model tested lagged its own advertised cutoff by three to four years, which rules out a single-model artifact. A companion paper, [Dated Data](https://arxiv.org/abs/2403.12958), formalises the gap between reported and effective cutoffs and traces it to temporal bias in CommonCrawl and to deduplication failures.

Instruction tuning narrows this considerably — ChatGPT in that study peaks at 2021 — so “deployed assistants are three years behind” would be an overstatement of what was measured. The durable finding is the mechanism, which the EMNLP survey expects to intensify rather than resolve, since it is a property of pretraining on collected data and the three available mitigations — knowledge editing, retrieval and continual learning — are, in their phrase, not magic bullets.

Scale does not rescue it either. [FreshQA](https://arxiv.org/abs/2310.03214), from Google and UMass Amherst, split 600 questions into never-changing, slow-changing and fast-changing categories and collected more than 50,000 human judgements. Without a search engine attached, no model exceeded 15% on fast-changing questions in either grading mode. Increasing T5 from 770M to 11B parameters roughly tripled accuracy on never-changing knowledge, from 5.6% to 19.0%, while fast-changing accuracy went from 0.8% to 2.4% — effectively flat. PaLM from 8B to 540B showed the same dissociation. Those figures come from a single run in April 2023 against models with 2021 cutoffs, so treat them as historical measurements of a mechanism rather than as current model properties. The mechanism is the part that generalises: parameters buy you static knowledge, and buy you almost nothing on facts that move.

Which leaves the corpus. If the model's internal knowledge is structurally behind and cannot be scaled forward, the currency of the system is whatever you retrieve into it.

## The infrastructure layer has already started absorbing this

The clearest evidence that freshness has stopped being a content problem is that it is being implemented as plumbing by the companies who build the plumbing.

Microsoft shipped [freshness-aware retrieval](https://learn.microsoft.com/en-us/azure/search/agentic-retrieval-how-to-configure-freshness) in the Azure AI Search 2026-08-01 preview API. A knowledge source now carries a *freshnessPolicy* with a *boostingDuration* in ISO 8601 form, so *P90D* biases ranking toward the last ninety days. The documentation is careful that this is “a ranking bias, not a hard filter,” recommends it for release notes, policy updates, runbooks and service advisories, and closes with the line that matters most: if ranking does not behave, inspect the *last_modified* field, because “missing, stale, or inconsistent date values reduce the quality of the freshness signal.” The infrastructure will weight recency for you. It cannot invent the metadata that tells it what recent means.

Google's Vertex AI Search documents the other half — the refresh itself — with unusual candour. After initial indexing it recrawls existing pages “on a best-effort basis,” and it “regularly refreshes data stores that encounter a query rate of 50 queries/30 days.” Manual recrawl is capped at 20 calls per day per project and 10,000 URIs per call, timing out after 24 hours. Read that as an operational statement: automatic refresh is neither guaranteed nor uniform, and it is partly a function of how much traffic a store gets. The pages nobody queries are the pages least likely to be refreshed, which is precisely backwards from where stale content does its damage.

Glean approaches it from the human side. Its [document verification](https://docs.glean.com/user-guide/knowledge/verification/how-verification-works) attaches a badge recording who verified a document and when, lets anyone request verification from an owner, and pushes re-verification reminders back into the owner's task list on expiry. That is a named person, a timestamp and a recurring obligation — a maintenance loop rather than a ranking signal.

And in June 2026, Google Cloud published the [Open Knowledge Format](https://github.com/GoogleCloudPlatform/knowledge-catalog/blob/main/okf/SPEC.md), a specification for sharing organisational knowledge with AI agents. Its Provenance, Trust and Lifecycle field family is the most direct statement of the argument that exists. *verified* records a list of *{by, at}* events and derives a trust tier from unverified through machine-confirmed to human-reviewed. *status* takes *draft*, *stable* or *deprecated*, defaulting to stable when absent. And *stale_after* is an absolute instant, with the semantics stated plainly: a concept is stale when *now* is at or past *stale_after*.

An expiry date is now a field in a Google Cloud knowledge specification. Whatever else one thinks of the discipline, the question of whether a piece of organisational knowledge has gone off has been given a place to live in the schema — which is roughly the definition of something becoming infrastructure.

The distinction underneath all four was put well by Vivek Venkatesan of Vanguard, writing in [TechArena](https://techarena.ai/content/knowledge-freshness-the-missing-discipline-in-enterprise-ai) in June 2026: “Data freshness answers a familiar question: did the latest data arrive? Knowledge freshness asks a harder question: is the information an AI system is using still valid, authoritative, and safe to apply in this context?” Pipelines answer the first. Nothing in the standard stack answers the second.

## Where this argument runs out

Stated here so it cannot be quietly dropped later.

- **This is severity, not prevalence.** HoH, ClashEval, DRAGged into Conflicts and RAMDocs all deliberately inject or enrich conflict. Together they establish how badly systems fail when stale content is present. Not one of them establishes how often that happens in a real knowledge base. Anyone who says freshness is not their bottleneck is making a prevalence claim, and this literature can neither confirm it nor refute it.
- **None of it was measured on an enterprise corpus.** Wikipedia snapshot diffs, live search results and entity-ambiguity sets are not Confluence. Applying these percentages to an internal wiki is a reasonable extrapolation and an explicit one. It is not a measurement of your systems.
- **Ranking is a comparably powerful lever.** In HoH, passage ordering alone swings Llama-3.1-70B's score by up to 25% and Qwen-2.5-7B's by more than 50% — the paper's worst result, below random, comes from a particular date-descending ordering rather than from the corpus. The authors still conclude that ordering “cannot fully mitigate” the problem, so retrieval-side controls are a partial remedy, not a substitute for corpus hygiene. Which of the two buys more accuracy per unit of effort has not, as far as we can find, been measured on the same corpus by anyone.
- **We could not verify the business case.** We searched specifically for credible surveys quantifying the prevalence or cost of outdated internal documentation and found nothing that survived checking. If you have seen a defensible number for what knowledge decay costs an organisation, we would like to see it, because we could not produce one.
- **The freshness statistics in circulation do not have sources.** The widely repeated figures — that content updated within thirty days earns more AI citations, that platforms have particular citation half-lives — recur across vendor blogs without a traceable primary study behind any of them. We are not going to repeat them, and neither should anyone selling on the back of them.
- **The numbers are dated.** These results were measured on 2023-era and 2024-era models. Follow-up work through 2025 and 2026 reaffirms the mechanisms, but nobody appears to have re-run HoH or FreshQA against current frontier models. The magnitudes should not be projected forward.

## What we are doing about it

Two things, both checkable.

The first is the [Stale Answer Benchmark](https://knowledgecompany.ai/blog/ai-outdated-company-information) — sixteen companies, four claim types each, three answer engines — whose method we published before collecting a single data point, so the design cannot be tuned to the result. It measures whether public AI assistants state a company's current published facts, which is the question every visibility dashboard skips in favour of counting mentions.

The second is this page. Its three dates now carry three different meanings: when it was first published, when it was last rewritten, and when its claims were last checked against their sources. The reading-time label is computed from the word count rather than estimated. If any of that drifts again, it will be a finding about us, and we would rather publish it than have it found. Knowledge Company maintains this page under the same discipline it sells: every figure above carries a source and a date its verification expires, listed in the public register at knowledgecompany.ai/claims.

## More from the knowledge library

### AI Systems Are Serving Outdated Company Information

The Stale Answer Benchmark: a published method for testing whether AI assistants state current facts about a company, released before we collect the data.

### Different AI Engines Give Different Answers About the Same Company

What has actually been measured about cross-engine inconsistency, what has not, and why some of the disagreement starts in your own published corpus.

---

## Related documents in this mirror

- [Blog](https://llms.knowledgecompany.ai/blog.md)
- [A Field Guide to Claim Decay](https://llms.knowledgecompany.ai/blog/field-guide-claim-decay.md)
- [Introducing The Knowledge Company](https://llms.knowledgecompany.ai/blog/introducing-the-knowledge-company.md)
- [AI Systems Are Serving Outdated Company Information](https://llms.knowledgecompany.ai/blog/ai-outdated-company-information.md)
- [Different AI Engines Give Different Answers About the Same Company](https://llms.knowledgecompany.ai/blog/conflicting-ai-answers.md)
- [9 GEO Tools, 4 Measurement Methods](https://llms.knowledgecompany.ai/blog/geo-tools-what-they-measure.md)
- [What 9 GEO Tools Actually Cost in 2026](https://llms.knowledgecompany.ai/blog/geo-tools-pricing.md)
- [9 GEO Tools Track Whether AI Mentions You. Almost None Check Whether It’s True.](https://llms.knowledgecompany.ai/blog/geo-tools-accuracy-gap.md)

*This document is the markdown mirror of https://knowledgecompany.ai/blog/knowledge-freshness-ai-infrastructure, published by Knowledge Company -
a knowledge freshness platform that detects, verifies and updates outdated claims in company content.*

*Mirror index: https://llms.knowledgecompany.ai/index.md - Generated 2026-09-02*