Metadata Is the Unsung Hero of RAG Accuracy: What to Extract and How to Filter

Short answer: Attach three classes of metadata to every chunk — structural (page, section, headers), content-based (entities, keywords, summary), and contextual (source, date, sensitivity, language). Then use those fields as pre-filters before similarity search. This is what turns a single-axis vector lookup into multi-dimensional, filtered retrieval, and it is the cheapest accuracy win most RAG systems leave on the table.

Why metadata moves the accuracy needle

A pure vector search ranks chunks on one axis: semantic similarity to the query. That is blind to whether a chunk is from last week or five years ago, from a trusted source or a scratch doc, in the right language, or even the right document type. Metadata adds those axes. As Unstructured puts it, metadata converts "single-axis similarity search into multi-dimensional, filtered search." You stop asking only "what looks similar" and start asking "what looks similar and is recent and is from this source."

The practical effect: pre-filtering by date, source, or type removes irrelevant candidates before ranking, so the top-k you feed the LLM is denser with correct context.

What to extract: the three metadata classes

There are three classes worth capturing per chunk. Grab the cheap ones from your parser first.

Structural and contextual metadata fall out of parsing and file handling for essentially nothing. Content-based metadata is the only class that needs a model, so it is the only place you should be weighing cost.

From similarity search to filtered search

Filters are applied as constraints alongside the vector query. In practice you split query intent into a semantic part and a structured part:

# Structured filter narrows candidates; vector search ranks within them.
results = collection.query(
    query_vector=embed("Q2 revenue recognition policy"),
    filter={
        "doc_type": "policy",
        "language": "en",
        "modified_date": {"$gte": "2024-01-01"},
        "sensitivity": {"$in": ["public", "internal"]},
    },
    limit=8,
)

The filter shrinks the candidate set; similarity then ranks what survives. One caveat: filter performance depends on indexing. Well-indexed payload filters usually speed up search by shrinking the set the vector engine has to score. Unindexed filters force a scan and can slow things down — so index the fields you filter on.

Metadata for citations and grounding

Metadata is also how you make answers verifiable. The Databricks RAG cookbook is explicit that chunks enriched with source id, page range, and section title are what enable citations. Store those three fields on every chunk and your generator can emit "per §4.2, p. 11 of policy-2024.pdf" instead of an ungrounded claim. This is the difference between an answer a user can trust and one they have to re-verify by hand — and it comes almost entirely from structural + contextual fields you already extracted for free.

Cost note: LLM metadata vs deterministic extraction

Do the deterministic extraction first. Page numbers, headers, section titles, source ids, dates, and language come straight from parsing and cost nothing per chunk. Reserve LLM calls for the content-based class — entity extraction and summaries — where they genuinely add signal. On a large corpus, running an LLM over every chunk to regenerate metadata you could have read from the parser is pure waste. A good rule: only pay a model for metadata you cannot derive mechanically.

Extract structural and contextual metadata for free, spend model budget only on content-based fields, index what you filter on, and you get sharper retrieval plus grounded citations at almost no marginal cost.

Related: more writing.