Permission-aware retrieval over enterprise documents

The problem

An organisation wants its staff to be able to ask questions of its own documents.

The documents already carry access controls, applied inconsistently across a document management system, a wiki, a shared drive, a ticketing system and an email archive. Some of those controls are deliberate and current. Others are the residue of a reorganisation three years ago that nobody has audited since.

A retrieval system built without reference to those controls flattens all of it. Every document goes into one index, and every user who can ask a question can surface any passage in that index.

The failure is not subtle and it does not take long to appear: someone asks a broad question about compensation policy and receives a fragment of an individual salary review, correctly retrieved and helpfully summarised.

What makes this failure mode particularly awkward is that it is invisible during development.

A pilot built on a curated set of a few hundred non-sensitive documents behaves perfectly. The problem arrives with the corpus, at the point where the system is connected to real sources and real users.

That is also the point at which it has become visible to enough people that rolling it back is a conversation with someone senior.

When to use this

• Source documents carry meaningful access restrictions, and those restrictions matter to someone who could stop the project.
• Permissions already exist in a system of record that can be queried programmatically — a directory service, the source application’s own API, or both.
• An audit trail of who retrieved which document is a requirement, or is likely to become one.
• The corpus is large enough that manual curation is not a viable alternative.
• Users span more than one permission tier. A system where every user has identical access does not need this pattern.

When not to use this

Every user can legitimately see every document

Plain retrieval is simpler, faster and cheaper. Adding permission filtering here buys nothing and costs latency on every query.

The permission model is undocumented or disputed

This is the important one. The pattern encodes an access model; it does not improve one.

Point it at a permission structure nobody can explain and it will faithfully and efficiently reproduce every error in that structure, while adding a natural-language interface that makes those errors easier to find.

The corpus is small and stable

For a few hundred documents that change rarely, a hand-curated index with a fixed audience may be simpler and more appropriate.

Permissions change minute to minute and staleness is unacceptable

The snapshot-at-ingest approach described below assumes eventual consistency measured in minutes.

Where enforcement must be exact at the instant of the query, the permission check has to happen against the system of record at query time, which is a different design with materially worse latency.

The requirement is really a data loss prevention requirement

If the concern is staff exfiltrating documents they already have access to, this pattern is orthogonal to the problem and will not help.

Architecture

Permissions are captured at ingestion time, alongside the document content, and carried as metadata on every chunk derived from that document.

At query time the requesting user’s effective permission set is resolved — expanded through group membership, including nested groups — and applied as a pre-filter on the vector search.

Restricted content never enters the candidate set.

The tempting shortcut is to retrieve first and filter afterwards, discarding the chunks the user should not see. It is easier to build, it performs better, and it is wrong.

By the time filtering happens, the restricted content may already have been loaded into the context assembled for the model.

A model that has read a document can leak its contents through paraphrase, summary, or an inference that does not quote it directly.

Post-filtering protects the response body while leaving the actual exposure intact. Treat it as a bug, not an optimisation.

Components

Tradeoffs

Implementation notes

Resolve nested group membership

Resolve group membership transitively and cache the result. Nested groups are normal in any organisation old enough to have restructured.

A naive per-query directory lookup that walks the tree can dominate query latency.

Cache with a short TTL and accept the resulting staleness window deliberately rather than by accident.

Treat permission drift as a monitored data-quality problem

The re-sync job is the component that gets deferred and the component that causes the incident.

Permissions drift: people change teams, projects close, contractors leave, and a document’s ACL is tightened after the index was built.

Treat permission drift as a monitored data-quality problem with alerting and an owner, not as a nightly cron job that nobody reads the output of.

Build an adversarial test set

Build an adversarial test set before building the system and run it in CI.

At minimum, include:

• A user who should see nothing at all.
• A user with access to exactly one document in the corpus.
• A user whose access to a specific document was revoked after ingestion.
• A user who is a member of a group nested three levels deep.

Each is a distinct failure mode and each has a different root cause.

Decide what happens when filtering removes every result

Decide early what the system does when retrieval returns nothing because everything relevant was filtered out.

The honest answer — that there may be material on this topic the user cannot see — is itself an information disclosure in some organisations, while a silent “no results found” can be confusing in others.

This is a policy decision that needs an owner outside engineering, and it is better made before launch than during an incident.

Log the permission filter

Log the filter that was applied, not just the results.

When someone eventually asks why a particular document did or did not appear, the answer lives in the filter expression.

Reconstructing it after the fact from a results log is close to impossible.

References

• Document-level access control in Azure AI Search — Microsoft’s overview of approaches including security filters and native ACL preservation.
  Microsoft Learn
• Security filters for trimming results in Azure AI Search — implementation guidance for filter-based security trimming.
  Microsoft Learn
• azure-search-openai-demo: login and ACL documentation — reference implementation covering authentication and document-level ACL behaviour.
  GitHub
• GPT-RAG: authentication and document-level security — guidance for user, group and RBAC-scope metadata on indexed documents.
  GPT-RAG documentation