Relevance
Measure whether retrieved chunks actually answer the user question and include decisive evidence early enough.
RAG evaluation
Fix retrieval quality before blaming the model.
RAG evaluation should measure whether the retriever gave the model a fair path to the answer. Vectory checks search progression, evidence diversity, source quality, and synthesis timing across production traces.
Direct answer
A RAG evaluation should score retrieval and answer quality side by side. If retrieved evidence is thin, stale, duplicated, or off-topic, a better model will not reliably fix the answer.
Diversity
Flag duplicate source families and reward independent sources that change what can be known.
Faithfulness
Check whether the final answer uses the strongest evidence and avoids unsupported claims.