Building Agentic Search on Qdrant: A Production Architecture Walkthrough

Short answer: Put an LLM agent in charge of tool calls, where each tool does exactly one Qdrant query combining vector similarity with payload pre-filters, then hydrates the rich fields from Postgres and post-sorts in code. Keep each vector query under ~50ms so the agent can loop without becoming brittle, enrich queries to match how your documents were embedded, and relax filters progressively when a query returns nothing.

I run this architecture in production for a food-discovery platform. Here is what actually holds up.

The core loop: one tool, one Qdrant query

The agent (Gemini 2.5 Flash over an OpenAI-compatible API) sees a set of narrow tools — search_dishes, search_restaurants, find_pairings — and decides which to call with which arguments. Each tool does the same shape of work:

• One Qdrant call: vector similarity plus payload pre-filters in a single request.
• Hydrate the shortlisted IDs from Postgres to get the heavy fields.
• Post-sort and dedupe in Python before handing results back.

Keeping tools single-purpose matters more than it sounds. A tool that does one Qdrant query is easy for the model to reason about, easy to retry, and easy to bound in latency. Fat tools that fan out into multiple queries make the loop unpredictable and slow.

Query enrichment: match the document format

Vector recall collapses when the query and the stored documents don't look alike. I embed documents in a structured shape — text | Cuisine: X | Tags: Y — so I enrich the agent's raw query into the same shape before embedding it. A bare "something spicy and cheap" recalls far worse than the enriched form that carries the same cuisine and tag scaffolding the documents were written with.

Embeddings are OpenAI text-embedding-3-small at 512 dimensions, cached in Redis for 7 days. The cache is not a nicety — in an agent loop the same or near-same query gets re-embedded constantly, and recomputing every time wrecks both latency and cost.

Pre-filters vs post-filters, and the MatchAny gotcha

Push everything you cheaply can into Qdrant payload pre-filters — cuisine, price band, availability. These shrink the candidate set before vector scoring, which is what keeps queries fast. Reserve post-filters in Python for fields you don't index or derive at read time.

The gotcha that cost me real debugging: multi-value filters. To match "Italian OR Japanese," use a single MatchAny on the cuisine field. Stacking two MatchValue conditions ANDs them, so you silently ask for dishes that are both Italian and Japanese — and get nothing. MatchAny is OR; separate conditions are AND. Also note MatchText needs a text index on the field, or it just won't match.

Zero-result handling: progressive relaxation

Agents will construct over-constrained queries. Rather than returning an empty list and letting the model flail, the tool relaxes filters progressively and retries:

filter_tiers = [
    all_filters,                 # cuisine + price + tags + dietary
    drop(all_filters, "tags"),   # loosen soft constraints first
    drop(all_filters, "price"),  # then numeric bands
    {"cuisine": cuisine},        # keep only the hard intent
]
for f in filter_tiers:
    hits = qdrant.query(vector=vec, query_filter=f, limit=k)
    if hits:
        return hydrate(hits)
return semantic_only(vec)  # last resort: pure vector, no filters

Relax soft constraints (tags) before hard ones (cuisine), and fall back to a filter-free semantic search only as the final tier. This keeps one bad argument from turning into a dead end the agent has to reason its way out of.

Latency budget: why <50ms per vector query matters

Qdrant's own agentic guidance puts the target at under 50ms per vector query. That number is about the loop, not a single request — an agent may issue several tool calls per turn, so 50ms each is the difference between a snappy answer and a multi-second stall that makes the whole system feel brittle. Pre-filters, a lean payload, and small vectors (512 dims) are how you hit it.

The same low-latency profile pays off for agent memory: Qdrant supports real-time upserts, so an agent can write what it learns mid-session and query it back without a rebuild step.

Build narrow tools, filter in Qdrant, hydrate in Postgres, and defend the latency budget — the agent's reasoning is only as good as how fast and clean each query comes back.

Related: more writing.