/4 min/reference
Separating Retrieval Failures From Generation Failures in RAG Evaluation
Score retrieval and generation separately: the pair of metrics localizes whether a bad RAG answer is a retriever or an LLM problem.
Topic
LLM Evaluation & Observability
Evals, hallucination detection, and monitoring so you know when the system is actually working.
/3 min/reference
Reference-Free vs Reference-Based LLM Evaluation: When You Actually Need Ground Truth
Reference-based evals compare output to a known correct answer; reference-free evals score intrinsic properties like grounding without ground truth.
/4 min/reference
Prompt Versioning in Production: Git vs Prompt Management Tools, and the Eval Gate That Prevents Silent Breaks
Version prompts in Git for small eng teams, move to a prompt tool when non-engineers contribute, and gate every change on an eval suite.
/2 min
The single most useful log line in a RAG system
If you log one thing in your RAG pipeline, log the final assembled prompt. It's the fastest path from 'bad answer' to root cause.
/4 min/reference
Building a Golden Dataset for RAG Evaluation From Your Production Traces
Mine failing production traces, store input plus retrieved chunks plus expected behavior, and version 50-100 focused cases in-repo.