Fix retrieval quality before blaming the model

When a RAG or research agent gives a weak answer, the model is usually the first suspect. Sometimes that is right. But in many production traces, the model is answering from thin, stale, duplicated, or conflicting evidence. Better prompting will not fix a retriever that never found the relevant source, and a larger model will not reliably rescue context that points in the wrong direction.

Teams need to evaluate retrieval as a first-class part of the agent, not as an implementation detail hidden behind the final answer. The question is not only whether the answer was correct. The question is whether the agent had a fair path to the answer.

Search is useful when each round changes the information state. A good second search narrows a vague question, checks a conflicting claim, finds a primary source, or fills a missing field. A bad second search repeats the same query, opens near-duplicate results, or collects more text without updating the working answer.

This is why retrieval evaluation should include time and sequence. Aggregate recall is not enough. You want to know when the decisive evidence appeared, how many rounds it took, and whether the agent recognized that it had enough information to answer.

Long context can hide weak retrieval. Ten chunks from the same page are not the same as three independent sources that triangulate a claim. For enterprise data, diversity may mean different systems: CRM notes, support tickets, policy docs, billing records, and meeting transcripts. For web research, it may mean primary sources, recent updates, documentation, and trusted analysis.

An agent should be rewarded for collecting evidence that changes what can be known, not for filling the context window. This is especially important when agents perform multi-step work, because early retrieval mistakes can cascade into planning and execution.

A synthesis checkpoint is a moment where the agent states what it knows, what remains uncertain, and what it will search next. It prevents search from becoming motion. It also creates an evaluation artifact: reviewers can see whether the agent understood the evidence before moving on.

In many failing traces, the agent searches repeatedly and only synthesizes at the end. By then, it may have mixed stale context with decisive context, or it may have lost the original question. A short checkpoint after important retrieval rounds keeps the run grounded.

A retrieval score labelled quality tells a reviewer nothing about where to work. The open-source Ragas library publishes metric definitions that separate the layers cleanly, and borrowing its vocabulary costs nothing and makes a report legible to people outside the team. Its retrieval metrics include Context Precision, Context Recall, Context Entities Recall, Noise Sensitivity, Response Relevancy, and Faithfulness.^[1]

Two of those carry definitions worth stating exactly, because they answer different questions. Context Precision measures the degree to which relevant chunks in the retrieved context are placed at the top of the ranking, so it is sensitive to order rather than only to membership. The Ragas documentation shows what that sensitivity looks like: moving an irrelevant chunk from position two to position one drops the score from about 1.0 to about 0.5, while leaving it at position two does not change the score at all.^[2] A reranker regression shows up here before it shows up anywhere else.

Faithfulness measures the other end of the pipeline. A response is faithful if all its claims can be supported by the retrieved context, scored as supported claims divided by total claims on a 0 to 1 scale, computed by extracting discrete claims, verifying each against the context, and dividing. The documented worked example makes the granularity clear: an answer giving a country and the date 20 March 1879 against a context giving 14 March 1879 decomposes into two claims, one of which holds, for a score of 0.5.^[3] A single wrong detail inside an otherwise correct paragraph is visible instead of averaged away.

Pairing the two is what makes the report diagnostic. High Context Precision with low Faithfulness points at synthesis or the prompt, because the evidence was there and in the right order and the answer still drifted. Low Context Precision with high Faithfulness means the model stayed honest about thin evidence, and the retriever is the thing to fix. Both low is a retrieval problem wearing a model problem as a costume, which is the case this article exists to catch.

Before replacing the model, compare retrievers, chunking strategies, metadata filters, rerankers, and query generation. Keep the answer model fixed while you test retrieval changes. Then keep retrieval fixed while you test model changes. This prevents teams from attributing gains to the wrong layer.

A mature evaluation report should show retrieval metrics and answer metrics side by side. If retrieval quality is low, the model score is not the root cause. If retrieval quality is strong and the answer is still weak, then prompt, synthesis, or model capability is the right next investigation.

Grounding failures are common enough to have their own labelled corpora, which is useful when you want to test a detector rather than trust one. RAGTruth annotates close to 18,000 naturally generated RAG responses at both case level and word level, including hallucination intensity, and its authors report that fine-tuning a relatively small model on that data reached performance competitive with prompt-based detection using frontier models.^[4] If you are building a faithfulness checker, measure it against labelled data before it starts gating releases.