/4 min/reference
Separating Retrieval Failures From Generation Failures in RAG Evaluation
Score retrieval and generation separately: the pair of metrics localizes whether a bad RAG answer is a retriever or an LLM problem.
Tag
#observability
3 posts.
/2 min
The single most useful log line in a RAG system
If you log one thing in your RAG pipeline, log the final assembled prompt. It's the fastest path from 'bad answer' to root cause.
/4 min/reference
Building a Golden Dataset for RAG Evaluation From Your Production Traces
Mine failing production traces, store input plus retrieved chunks plus expected behavior, and version 50-100 focused cases in-repo.