Permission-Aware RAG: Access Control Belongs Inside Retrieval.

The finding we write up most often

An internal assistant, built well by competent engineers, indexed the company's documents into a vector store and answered questions with citations. It worked. Then someone in support asked a question phrased just right and got a paragraph from a document they had no business reading.

Nothing was "hacked." The retrieval layer simply had no concept of who was asking. Every chunk in the index was equally reachable by every query, and the only thing standing between a user and a restricted document was whether their phrasing happened to rank it in the top five.

This is the most common serious finding in enterprise RAG, and it is structural. Teams get authentication right at the application edge, then build a retrieval layer that quietly bypasses every permission model the organization spent a decade building.

One rule, and where teams break it

The model must never receive a chunk the user is not authorized to read. Everything below follows from that.

The subtle failure is post-filtering: retrieve the top k by similarity, then drop the ones the user cannot see. This feels equivalent and is not. It leaks in three ways. If you filter after generation, the model already conditioned on restricted text and will paraphrase it. If you filter after retrieval but before generation, an authorized-but-mediocre answer silently degrades because the good chunks were discarded, and users learn that certain topics return nothing, which is itself an existence oracle. And the number of results returned varies with what exists, which leaks the shape of the corpus.

Pre-filtering means the permission predicate is part of the query, evaluated by the vector store, so the candidate set only ever contains authorized chunks. Every serious vector store supports this: metadata filters in Pinecone and Qdrant and Weaviate, WHERE clauses in pgvector, filter contexts in OpenSearch and Elasticsearch. Use them.

def retrieve(query: str, principal: Principal, k: int = 8):
    # The ACL predicate is part of the search, not a step after it.
    # `allowed_groups` is denormalized onto every chunk at index time.
    return store.search(
        vector=embed(query),
        k=k,
        filter={
            "tenant_id": {"$eq": principal.tenant_id},
            "allowed_groups": {"$in": principal.group_ids},
            "classification": {"$lte": principal.clearance},
            "deleted": {"$eq": False},
        },
    )

Two details worth stating explicitly. The tenant_id predicate is not optional even when your code "always" sets it, because the one code path that forgets is the incident. Enforce it in a wrapper that no caller can bypass, and make the raw client private to that wrapper. And principal comes from the authenticated session, never from a request parameter and never from anything the model produced. An agent that can choose its own group_ids has no access control at all.

The index is a cache of permissions, and caches go stale

Denormalizing ACLs onto chunks makes queries fast and creates a consistency problem: the moment someone is removed from a group, your index is wrong, and it stays wrong until something re-syncs it.

For most teams the honest answer is a hybrid. Denormalize for the filter, then verify before the text reaches the model:

hits = retrieve(query, principal, k=20)          # fast, filtered, possibly stale
doc_ids = {h.doc_id for h in hits}
live = authz.bulk_check(principal, doc_ids)       # source of truth, one batched call
chunks = [h for h in hits if live[h.doc_id]][:8]

Over-fetch, verify against the real authorization service in one batched call, then truncate. The check costs a few milliseconds and closes the staleness window entirely. If your authorization service cannot answer a bulk check quickly, that is worth fixing regardless of AI.

Alongside that, drive index updates from permission-change events rather than a nightly job, and treat the reconciliation sweep as a control with an owner: it should log how many chunks it corrected, and a sudden spike means something upstream is broken.

Deletion deserves its own paragraph because it is where "we removed it" turns out to be false. A document deleted at the source lives on in the vector index, in any cached retrieval results, in conversation transcripts that already quoted it, and quite possibly in a nightly snapshot. Deletion has to fan out to all of them, and under GDPR or a client's data-return clause, being able to prove that fan-out is the actual requirement.

Embeddings are the documents, legally and practically

A persistent misconception is that a vector store holds "just math" and therefore sits outside the controls that apply to the source documents.

That is not true. Morris et al. demonstrated that text can be reconstructed from dense embeddings with high fidelity, perfectly recovering a large share of short passages, and follow-up work has shown reconstruction of names from clinical notes. An embedding is a lossy but often reversible encoding of the text.

The consequence is simple and load-bearing for your architecture: the vector store inherits the classification of everything indexed into it. Same encryption requirements, same access controls, same residency constraints, same retention and deletion obligations, same audit scope. A team that would never copy client documents into an unreviewed SaaS tool should apply exactly that scrutiny to a hosted vector database, because that is what they are doing.

Tenant isolation: filter or separate

For multi-tenant systems, a metadata filter is a logical boundary enforced by application code. Separate collections, namespaces, or databases per tenant are a physical boundary enforced by infrastructure.

Shared index with filters is cheaper and scales to many tenants, and it is one forgotten predicate away from cross-tenant disclosure. Per-tenant isolation costs more overhead and makes cross-tenant leakage require a genuine infrastructure failure rather than a code mistake.

Our default: shared index with enforced filters for internal, single-org deployments; physical separation whenever tenants are separate legal entities, whenever the data is regulated, and always for the top tier of clients who asked how isolation works. When a client's risk team asks that question, "a WHERE clause" is a materially weaker answer than "a separate encrypted collection with its own key."

Testing it, in CI, every build

Access control that is not tested continuously is access control that used to work. Three tests catch nearly everything:

Canary documents. Seed each tenant and each permission tier with a unique, unmistakable document ("QUARTERLY-CANARY-7F3A, tenant B, restricted"). In CI, query as tenant A with phrasings designed to surface B's canary. Any hit fails the build. This is the single highest-value test in the suite.

Principal fuzzing. For a matrix of principals and queries, assert that returned doc_ids are a subset of what the authorization service independently says that principal can read. Discrepancies are findings.

Revocation timing. Remove a user from a group, then immediately query. Assert the document is unreachable. This measures your actual staleness window rather than the one in the design doc.

Wire these into the same gate that governs model promotion, with a hard floor and no regression tolerance: a retrieval leak is not a quality metric to trade against helpfulness.

Log retrieval, not just generation

Most teams log the prompt and the answer. For an audit, that is half a record. You also need which chunks were retrieved, which documents they came from, which principal asked, and which authorization decision applied.

When someone asks "did the assistant ever expose the acquisition memo," a generation log forces you to guess from paraphrases. A retrieval log answers it with a query. That log is also the thing that makes citations trustworthy, because you can prove after the fact that the answer was grounded in documents the user was allowed to see. It belongs in the same evidence architecture as the rest of your AI decision records.

The short version

Filter before you retrieve, never after. Take the principal from the session, never from the request or the model. Over-fetch and verify against the real authorization service. Treat the vector store as a copy of the documents, because it is. Isolate tenants physically when they are separate legal entities. Test with canaries in CI. Log retrievals.

None of this is exotic, and all of it is cheaper to build in week one than to retrofit after the assistant quotes a document to the wrong person.