Law 8 - When Feeding AI 'More Data' Only Reduces Output Quality
Contents
Pour everything into one index and retrieval drowns. A polluted index is worse than no index, because a confident answer from a stale source is more dangerous than no answer at all.
The most common mistake I see when people build a corpus is to overload it. Every document the business has ever produced goes into one folder, and the AI is expected to make sense of the pile. You have done the same thing to a Google Drive folder at some point, and you know what finding anything in it was like.
The number: more context, worse answers
Elixir Data’s work on what it calls context pollution names the problem directly. For AI systems, more information does not mean better decisions. Irrelevant or conflicting context lowers accuracy, whether it arrives as sheer volume, as accurate material that misleads in this context, or as sources that disagree.
The retrieval research says the same thing in numbers. A University of Science and Technology of China study on retrieval-augmented pipelines (arXiv 2024, accepted at CIKM 2025) found accuracy improved as chunk size grew from 50 to 500 tokens. At 1,000 tokens it declined, because the volume of material diluted relevance.
That is one framework on one benchmark, so treat the shape of the curve as the finding rather than the exact numbers.
The chunking post walks through that curve.
Stale versions, constantly
There is no single dramatic incident behind this law, because it happens all the time, even now. The AI finds a version of a document that is out of sync with the canonical one, assumes it is relevant, and builds a plan on it.
Most often that is because the AI forgot to search the knowledge base at all. The fix in my own workflow is almost comic in its simplicity: a two-word reminder, “use the Brain”, and it searches and finds the right document.
That only works because the Brain is kept clean, and keeping it clean is a job that never finishes. Send the same reminder at a corpus full of drafts and superseded copies and it will confidently find the wrong one.
Why filtering isn’t the answer
Your signed master agreement and a half-finished meeting note are not the same thing. A retrieval system treats semantic similarity as value, so if the note uses the same words as your question, it surfaces with the same confidence as your canonical SOP.
Better filtering is the instinctive fix, and it breaks the moment document structures change or people stop tagging consistently.
The answer is separation. Two corpora: one small and canonical, kept clean by a human who decides what belongs there, and one large and low-signal where everything else goes. Your AI can propose what should move up, but a person makes the call.
The hard part is keeping it that way
I have not met a client or colleague who objected to keeping a knowledge corpus clean, because the value is obvious once stated. What goes missing is the discipline, sustained for years.
That discipline is what stops retrieval from becoming the thing the RAG explainer warns about: a system that surfaces bad documentation faster, with more confidence behind it.
A quick way to see where you stand is to open the folder your AI indexes, pick any process your business runs, and ask whether you can point to the one canonical document for it. If the answer is “there are four spreadsheets and I’m not sure which is current”, your index is polluted, and the AI is no surer than you are. It just sounds sure.
FAQ
What counts as high-value knowledge? The documents you would want the AI to treat as the last word: current SOPs, signed agreements, the decision record and its reasoning, the specs your team actually works from. Drafts and working notes belong in the separate, larger store.
Who decides what goes in the canonical corpus? A human. The AI can propose candidates, and it should, but promotion into the canonical store is a decision, not an automation. That is the governance the whole law rests on.
What happens to the old versions? They leave the canonical index and go to the low-signal store if you want the history. Only one version of each document should be where the AI looks first.
Does the AI’s own output go into the canonical corpus? Only after review. Useful output is captured as a candidate, a person checks it, and if it holds it gets promoted.
Where do I start on day one? With the documents your team asks about most: the current SOPs, the price list, the agreements in force. Put the one current version of each into the canonical store and leave everything else where it is. A small set the AI gets right beats a large one it gets wrong.
This is the eighth of eight laws behind how an AI-native business gets built.