Why I don't use LangChain in production RAG

The standard advice is "start with LangChain, it's the default." And for a weekend prototype, that's fine — it gets you from zero to a working demo fast.

But the moment a RAG system has to run in production — real users, real data, real 2am pages — I've consistently found the abstraction costs more than it saves. So my production pipelines don't use it. Here's the reasoning.

The problem isn't LangChain. It's opacity.

A retrieval-augmented pipeline is, underneath, a very small number of steps: embed the query, search the vector store, assemble a prompt, call the model. The hard part isn't wiring those together — it's knowing exactly what happened when the output is wrong.

That's precisely what a heavy framework takes away from you.

When retrieval returns garbage, I want to answer three questions in seconds: what did the retriever actually return, what prompt did the model actually receive, and where did it diverge from what I expected? Inside a framework, the prompt is assembled somewhere I can't easily see, behind helpers and callbacks. So debugging becomes archaeology through someone else's abstraction instead of reading my own code.

What "direct" actually looks like

The whole retrieval-to-answer path, in plain sight:

# Embed → search → assemble → call. Nothing hidden.
query_vec = embed(query)                       # my embedding call
hits = store.search(query_vec, limit=8, filters=filters)
context = "\n\n".join(h.text for h in hits)
 
prompt = f"{SYSTEM}\n\nContext:\n{context}\n\nQuestion: {query}"
answer = llm.chat(prompt)   # I can log this exact string. That's the point.

It's more lines than a framework one-liner. That's a feature. Every line is one I can read, log, test, and change without guessing what a wrapper does under it.

The three things I stopped losing

• Visibility into the prompt. The single most useful log line in any RAG system is the final assembled prompt. When I own the assembly, I own that log.
• Debuggability. A bad answer traces to my code, not to a version of a dependency that quietly changed behaviour on upgrade.
• Control over cost and latency. Nothing runs that I didn't write. No surprise extra model calls, no hidden retries.

To be fair

LangChain is genuinely useful for what it's good at: prototyping, learning the moving parts, and exploring ideas quickly. If you're figuring out whether an idea works at all, reach for it.

The mistake isn't using a framework. The mistake is carrying that abstraction into production and hoping it holds when the abstraction is the exact thing standing between you and a fix.

For production RAG, I'll take boring, transparent code over a framework I can't see into — every time.

If you run RAG in production: are you still on a framework, or did you also end up dropping to raw code? I'm genuinely curious where people landed.