Deadwater

sept 18 2026 · updated sept 19 2026

By Jack Virag

Your AI found two versions of the truth. Now what?

Resolve conflicting product facts with scoped source authority, a named owner, a practical decision record, and checks that the correction reaches every dependent output.

9 min read
context-ossource-qualityai-workflowscontent-operations
Your AI found two versions of the truth. Now what?

Two citations can disagree. Adding a third doesn't necessarily help.

Your product brief says one thing. The help center says another. The AI finds both, writes a confident answer, and attaches a perfectly functional link.

The citation works. The claim may still be wrong for the customer you're talking to.

This is one of the places where a knowledge system has to do more than retrieve relevant text. Someone needs to settle what applies, preserve the qualification, and make sure the next run doesn't resurrect the argument.

First, make sure you're asking the same question

Take a fictional software company whose Growth plan may or may not include scheduled exports. The approved catalog says manual-only. A newer help article says scheduled exports are included.

There's also a launch note and a sandbox where the feature works. Plenty of evidence. Still no reliable sentence for the launch article.

The useful question is narrower: Do Growth-plan production workspaces created on September 18 include scheduled exports?

Plan, environment, date, and workspace cohort all matter. “The feature exists” doesn't answer who can use it.

Source What it says What it doesn't establish
Approved catalog, revision 12 Growth has manual exports only Whether the latest rollout is reflected
Help article, revision 7 Scheduled exports are included on Growth Whether every workspace is eligible
Launch note, edited September 17 Broader rollout planned for October 1 Availability today
Sandbox observation, September 18 Scheduled exports work here Production eligibility

These details are invented to show the decision. In a real case, read the actual passages and the qualifications around them before building the table.

A help article may describe a beta. An announcement may promise next month's release. Summarizing away those conditions can manufacture a contradiction that wasn't there.

In this example, the sandbox and future launch note don't settle today's production availability. The catalog and help article still need reconciling. That's the disagreement to work on.

“Newest” is doing too much work

A September edit might fix punctuation in an August policy. A document written today might describe next month's plan. A fresh PDF timestamp might mean somebody downloaded an old file again.

The Dublin Core vocabulary distinguishes modification dates, validity periods, audience, and coverage. You don't need to implement the vocabulary to use the distinction.

Record when the claim applies separately from when the file changed. If the effective date is missing, that's a gap to resolve—not an invitation to borrow the timestamp.

Agreement can be misleading too. Three sales snippets copied from one obsolete brief are three copies of one assertion. They aren't independent witnesses.

This is part of the difference between a knowledge base and a Context OS. Finding a statement and deciding how to use it are separate responsibilities.

Search rank isn't authority

Semantic ranking orders results by relevance to a query. A highly relevant passage can still describe the wrong customer cohort.

Search filters can narrow candidates by indexed fields such as product, environment, or status. They help when the labels are accurate. They can't establish authority, repair bad metadata, or reveal evidence they excluded.

Don't label the inconvenient source “historical” just because it challenges the preferred one. A useful context strategy makes the disagreement inspectable. It doesn't make the search results politely agree.

Give the disagreement a decision-maker

Suppose this company's product owner, Maya, maintains plan-availability facts in the catalog. That gives the workflow a starting point and an owner.

It doesn't make catalog revision 12 infallible. A credible newer help article is a reason to investigate.

Authority depends on the question. The product owner may establish intended eligibility. The rollout owner verifies which production workspaces actually have the feature. The editor turns the confirmed facts into a readable explanation.

“Code beats docs” is too broad. Code can show implementation without proving deployment. A public page can show what was promised without proving current behavior.

GOV.UK's published fact-checking process separates subject expertise from content presentation and asks for specific corrections. That division is useful even if your whole team fits around one table.

Define authority in the workflow by the kind of fact being decided. Then send a question someone can actually answer:

Can we say Growth production workspaces created on September 18 include scheduled exports? Catalog revision 12 says manual-only; help revision 7 says included. Please confirm the eligible cohort and effective date against the rollout record.

Attach the passages. “Please approve this article” would bury the unresolved decision inside a much bigger task.

Hold the claim, not automatically the entire project

The workflow can continue drafting independent sections while the export claim waits. It shouldn't improvise “generally available,” average incompatible limits, or ask the model five times and treat the majority answer as a product decision.

Anthropic's hallucination guidance includes allowing uncertainty and checking source passages. Useful behaviors, but neither makes an obsolete source current.

Make the unresolved state part of the actual review process. If the disputed feature is the point of the page, hold the page. If it's incidental, an editor may remove that claim and publish an otherwise useful piece within the existing authority.

Removing the sentence doesn't resolve the source conflict. Keep that work open. A deadline also doesn't resolve it, however convincingly the calendar argues otherwise.

Leave something better than a chat thread

A small decision record needs:

  • The precise question and its scope.
  • The competing passages, source locations, and revisions.
  • The person responsible for resolving it.
  • The missing check or evidence.
  • The affected output and current release status.
  • The eventual answer, evidence, and decision date.

The W3C provenance primer describes how information relates to its revisions, creators, and activities. A file or issue can capture enough of that for this job. You don't need to build a knowledge graph before correcting a product sentence.

Preserve the disputed passages when the answer arrives. “Resolved” tells the next person much less than why one version was incomplete and where the approved answer applies.

From a good idea to a working system

A Context OS connects your company knowledge to repeatable work. See what goes into one.

Fix the source, then follow the copies

Now resolve the fictional case. Maya checks the rollout evidence with its owner. Growth workspaces created on or after September 15 have scheduled exports. Earlier workspaces remain manual-only, with a broader rollout planned for October 1.

The catalog missed the first cohort's release. The help article omitted the restriction. Neither source was an adequate answer on its own.

Catalog revision 13 and help revision 8 should state the eligible cohort, environment, and effective date. The launch note should keep the remaining rollout as a plan, not convert it into guaranteed availability.

Don't globally replace “manual exports.” That still describes older workspaces in this example. The qualification you just established is exactly what a broad find-and-replace could destroy.

Tie review to the changed version. GitHub's protected-branch controls can, when configured, dismiss stale approvals after relevant changes. A document revision and review record can serve the same basic purpose.

Approval of last week's wording isn't a review of today's changed claim.

The original document isn't the whole system

Find the briefs, FAQs, sales snippets, staged articles, and public pages that repeat the claim. Search for the feature and both wordings, not just a source ID that older copies may lack.

Check what the next AI run actually loads. A saved source pack or retrieval index may still contain the old passage after the maintained document changes.

Azure's indexer change-and-deletion guidance illustrates one trap: detecting updates and detecting deletions are different mechanisms. Your system has its own behavior. Inspect it.

For retrieval, ask the relevant question again and examine the returned passage and revision. Compare it with the maintained source or an independent current-revision inventory.

An old retrieved copy cannot prove its own freshness. If that's all you have, current availability remains unverified.

Keep these states separate:

State What you've established
Source corrected The maintained record has the scoped fact
Working inputs updated The next run loads the corrected material
Draft corrected The proposed output uses it correctly
Public claim verified The authorized release is visible at the destination

This is the maintenance work behind living documents for agents. Assign unfinished corrections to someone. A page waiting for release doesn't become fixed because the source now looks right.

Keep the awkward case

Once the conflict is resolved, it becomes a useful test. Anthropic's evaluation guidance recommends learning from real failures and defining when behavior should—and shouldn't—occur.

For this example, test more than the happy answer:

Case Expected behavior
New Growth workspace on September 18 Confirm using the corrected, scoped source
Workspace created before September 15 Preserve manual-only availability for now
Question about October 1 Describe the plan without guaranteeing it
Missing workspace cohort Ask or explain the conditional answer
Eligible sources still disagree Hold the claim and route the decision
Retrieved revision 12; inventory says 13 Fetch the current source or mark the claim unverified

Inspect the evidence as well as the final sentence. A model can guess the right answer while citing the wrong revision. That doesn't demonstrate a repaired source path.

Update the test when the actual rollout changes. Otherwise yesterday's correct answer becomes tomorrow's enforced mistake.

The same principle applies to editorial feedback: preserve the failure, the correction, and the limit of the rule.

If you want help building that correction path, bring one real disagreement and the places it has already reached. The useful system is the one that stops making you settle the same argument every Tuesday.

Put this to work

Bring a recurring content problem. We’ll help scope the system behind it.