Corrective RAG (CRAG) Explained: Self-Healing Retrieval Without Fine-Tuning

Short answer: Corrective RAG (CRAG) is a retrieval pattern that inserts a lightweight retrieval evaluator between your vector search and your LLM. The evaluator scores how relevant the retrieved documents actually are, then routes the request down one of three paths: proceed with good docs, filter and refine ambiguous ones, or fall back to an alternative source (like web search) when retrieval fails. It self-corrects bad retrievals before they reach the model, and it needs no fine-tuning.

What CRAG actually adds

Vanilla RAG blindly trusts whatever the retriever returns. If your top-k chunks are off-topic, the LLM either hallucinates or confidently answers from garbage. CRAG closes that gap by adding one evaluation step that classifies retrieved documents as Correct, Incorrect, or Ambiguous, and adjusts strategy accordingly (a framing echoed by Meilisearch and kore.ai).

The three actions

• Correct (proceed): Confidence is high. Pass the documents to the LLM as-is, optionally after a light knowledge-refinement/strip step.
• Ambiguous (filter + refine): The signal is mixed. Keep the strong chunks, drop the weak ones, and rewrite the query to sharpen the next retrieval. This is where CRAG earns most of its accuracy gains in practice.
• Incorrect (fall back): Nothing relevant came back. Trigger an alternative retrieval path — web search, an expanded index, or relaxed filters — rather than answering from a bad context.

The routing threshold is yours to tune. A common setup: two confidence cutoffs producing three bands.

Where the evaluator lives and how to score cheaply

The evaluator sits after retrieval and before generation. You have three practical options, cheapest first:

• Score reuse: Start from the retriever's own similarity scores. Free, but noisy — cosine distance is not relevance.
• Cross-encoder / reranker: A small reranker (e.g. a bge-reranker class model) gives a real relevance score per (query, doc) pair for a few milliseconds each.
• LLM judge: A tiny grader prompt ("is this document relevant to the query? yes/no + score") on a small, fast model. Flexible, but the most expensive option.

Keep the grader cheap. CRAG adds one evaluation per retrieval, and if you grade with your main model you can double response time and cost for no reason.

Why it fits closed APIs

The original Self-RAG line of work trains the model to emit reflection tokens — that means fine-tuning, which you can't do on GPT-4o or Claude. CRAG puts all the correction logic outside the model, so it works with closed models with no fine-tuning at all (customgpt.ai makes this point directly). A useful way to hold the distinction: CRAG improves the quality of the evidence; Self-RAG improves the model's reasoning over evidence (a comparison from elegant software solutions). If you're on a hosted API, CRAG is the pattern you can actually ship.

A minimal CRAG loop over Qdrant

async def crag_answer(query: str, k: int = 5) -> str:
    docs = await qdrant_search(query, k=k)          # vector retrieval
    graded = [(d, await grade(query, d)) for d in docs]  # small reranker/judge
 
    strong = [d for d, s in graded if s >= 0.7]     # Correct band
    weak   = [d for d, s in graded if 0.3 <= s < 0.7]  # Ambiguous band
 
    if strong:                                       # Correct: proceed
        context = strong
    elif weak:                                       # Ambiguous: filter + refine
        refined = await rewrite_query(query)
        context = weak + await qdrant_search(refined, k=k)
    else:                                            # Incorrect: fall back
        context = await web_search(query)
 
    return await llm.generate(query, context)        # closed API, no fine-tuning

grade() is the only new dependency, and it's swappable — reranker today, LLM judge tomorrow — without touching the loop. Log the verdicts; the distribution of Correct/Ambiguous/Incorrect over real traffic tells you whether your index or your threshold needs work.

Takeaway: CRAG is the cheapest reliable way to stop a hosted LLM from answering off bad retrievals — one grader, three routes, zero fine-tuning.

Related: more writing.