Keep restricted information out of unauthorised AI answers

What this means for your practice

Require the supplier to demonstrate the intended access boundary across retrieval and output. Review source permissions and derived responses together rather than assuming login alone provides adequate separation.

A practical conversation with your supplier

Ask the same fictional question using two roles with different access. Test a restricted sample document and then change one user's role. The supplier should show how the answer respects the intended boundary. Your responsible team should review both source access and what the user actually receives.

Plan the first improvement

Define access by actual staff responsibilities before choosing how the assistant presents answers. A person may be allowed to read general guidance but not a restricted management document. Ask the supplier to demonstrate the result across searching, answer text and any linked extracts, using fictional material and approved test accounts. Include a user whose access changes after earlier use. The practice should understand how the intended boundary is maintained in the deployed arrangement, without assuming a single successful login check proves it.

Keep source permissions under review as documents move between folders or teams. A source change can affect the assistant even when nobody edits its interface. Give staff an appropriate way to report a suspected access problem and name the responsible owner. The acceptance evidence should describe what each test role could and could not obtain. This makes the decision reviewable and avoids relying on a broad supplier assurance that the product has security features.

How to check the result

Count access-boundary failures in authorised testing.