Choose reliable sources for an internal AI assistant

What this means for your practice

Define which sources the assistant may use and how their status is represented. Start with a controlled information set and retain accountable ownership of the underlying guidance.

A practical conversation with your supplier

Ask a question whose answer appears only in an unapproved sample draft. The supplier should demonstrate how that situation is handled. Staff should be able to identify the approved supporting material and report a wrong answer. Adding more documents is not automatically an improvement if their status is unclear.

Plan the first improvement

Start with a narrow collection of guidance whose owners are known and whose status is clear. Ask which questions the assistant is meant to help staff answer, then select sources that actually support those tasks. A large shared drive may contain duplicates, old instructions and drafts that need sorting before they become useful reference material. Keep a route for source owners to withdraw or update content and ask the supplier how quickly the assistant reflects that change.

Test a withdrawn document as well as a new approved one. Staff should be able to find the supporting guidance and understand its scope rather than accept an answer simply because it arrived quickly. Assign someone to review reported mistakes with the source owner: sometimes the source itself is unclear, while other problems concern how the assistant used it. Keeping that distinction helps the practice make the right improvement and avoids rewriting prompts when the underlying procedure is the issue.

How to check the result

Track answers relying on obsolete or unapproved material.