1. Choose search or an assistant
Start by deciding whether people need documents or a synthesised response. Search returns relevant material for a person to interpret. An assistant can combine passages into an answer, but that extra step introduces opportunities to omit qualifications or state unsupported conclusions. A well-organised search interface may be enough for exact policies and forms.
Define the questions the service should handle. Internal policy lookup, technical documentation and support procedures each need different source boundaries. Decide whether answers are informational drafts or can be used directly in a customer interaction. Keep legal, financial and other regulated decisions with appropriately qualified people rather than presenting generated text as professional advice.
2. Understand retrieval-augmented generation
Retrieval-augmented generation, or RAG, finds relevant source material and supplies it to a language model when producing an answer. It does not simply teach the model every document permanently. This distinction matters because updates, deletions and access restrictions must be reflected in the retrieval system as well as in the original repository.
Documents are often divided into passages called chunks. Embeddings represent text numerically so that a system can compare similarity. Keyword search remains useful for exact references, product codes and unusual names. A hybrid approach combines keyword and semantic retrieval. Choose based on your questions and documents, not on an assumption that semantic similarity always retrieves the right evidence.
3. Prepare the source collection
Select authoritative material before indexing everything. A SharePoint folder may contain an approved policy, an older copy and someone’s working notes. Without ownership and status information, an assistant can retrieve all three. Preserve titles, headings, dates where relevant, document status and source links as metadata, meaning information that describes the document.
Scanned PDFs require optical character recognition, or OCR, to convert visible text into machine-readable text. Check tables, footnotes and multi-column layouts because reading order can change the meaning. Long tables may need a different representation from prose. Source preparation is not a cosmetic stage; retrieval cannot reliably recover information that was lost during extraction.
4. Preserve the access boundary
The assistant must not reveal material a user could not access in the source system. Apply access checks when retrieving passages, not merely when showing a final source link. Restricting links after confidential text has entered the model context is too late. Include group changes, deleted users and moved documents in the access design.
Conversation history also needs a policy. People may paste confidential information into questions even when the document collection is carefully controlled. Decide whether histories are retained, who can view them and how deletion works. Review provider contracts for processing locations, retention and permitted uses of submitted content. Do not assume every product tier has identical terms.
5. Evaluate evidence, not fluency
Create questions whose answers are known from the selected documents. Include questions that require several passages, questions with conflicting sources and questions with no supported answer. Check retrieval separately from generation: the right source can be missing, or it can be present while the model still misinterprets it.
Require citations that point to the relevant source passage where practical. A citation is not proof by itself; check whether it supports the statement beside it. Evaluate how the assistant expresses uncertainty and directs people to a document owner. A clear refusal to answer an unsupported question is preferable to a smooth explanation built from assumptions.
6. Assign ongoing ownership
A scoped project can include source selection, retrieval design, an interface, evaluation cases and maintenance instructions. Agree who approves additions, retires obsolete documents and reviews reported errors. Define what happens when a source is updated and how the search index is refreshed. These operational responsibilities remain after the interface is launched.
For an initial discussion, describe the intended users, the document repositories and the questions they need to answer. Share categories, not confidential files. Begin with data readiness if ownership or permissions are unclear. Use AI governance to establish acceptable use and escalation before extending the assistant to more consequential tasks.
