In retrieval-augmented generation, answer quality depends on both the evidence retrieved and how the model uses it. Diagnose these stages separately. A longer prompt cannot reliably recover a passage the application never supplied.
1. Create a small, representative evaluation set
Collect realistic questions with the supporting passages you expect to find. Include ordinary requests, ambiguous questions, and questions the available sources cannot answer. Record what a useful answer should contain, rather than evaluating only whether the writing sounds confident.
2. Inspect the evidence reaching the model
For each question, inspect the retrieved chunks and final context. Does the expected passage exist in the source collection? Did extraction preserve important tables or headings? Did the passage survive chunking, appear in retrieval, and fit in the context actually sent to the model?
3. Change retrieval based on observed failures
If useful material is missing, inspect query behavior, chunk boundaries, filtering, and ranking. Where the search platform supports them, compare keyword, vector, and hybrid retrieval against the same evaluation set. Reranking is another experiment to assess; none of these options guarantees an improvement for every collection.
4. Evaluate the answer separately
If suitable evidence reaches the model, check whether the answer follows that evidence and addresses the question. Groundedness and correctness are different: an answer may accurately repeat an outdated source or give a correct fact without support from the retrieved material. Test appropriate uncertainty when the supplied sources cannot answer the question.
5. Keep the comparison repeatable
Change one variable at a time. Record retrieval configuration, relevant model settings, source versions, context, output, and evaluation results. Re-run the same examples to see which failure cases improved and which regressed. Consider latency and operating cost alongside answer quality.
A useful diagnostic record: the question, expected supporting passage, retrieved context, final answer, and a short explanation of the failure. This is more actionable than a screenshot of an answer that merely “looks wrong.”
What to bring to a support session
A sanitized input/output example, the expected behavior, a brief architecture sketch, and your evaluation criteria. Use sample documents you are authorized to share.
Official references
Published 31 October 2026. Check changes against your own environment and your team’s process.