
To build a knowledge base chatbot your employees will use, give every source document a named owner. Load each document with its permissions and effective date. Have the chatbot answer only from the passages it retrieves, with a link to the exact section. Send every question it cannot answer from a source to the person who owns that topic. Put the chatbot in Slack or Teams, where people already ask each other, and test it on real questions before anyone relies on it.
Retrieval from your own documents reduces wrong answers, but it does not remove them. A Stanford and Yale study published in the Journal of Empirical Legal Studies in April 2025 tested AI legal research tools. The tools came from LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI-Assisted Research and Ask Practical Law AI). They use retrieval-augmented generation: they look up passages in their publishers' own legal databases and write the answer from them. The study found that each tool still hallucinated, giving a wrong or unsupported answer, between 17% and 33% of the time. An employee who acts on a wrong policy answer once has little reason to ask the chatbot again. The design work below goes into the cases where the right reply is a citation, a warning or a handoff.
A knowledge base chatbot is often called a RAG chatbot, after retrieval-augmented generation. It retrieves passages from your documents, then writes its answer from them. It has five parts. The model writes the reply in the fourth, and the first three decide what it has to work with.
The sources are the documents you trust, each with an owner. Ingestion is the step that loads them. It splits each document into passages and keeps each passage's heading, owner, effective date and access list attached. The index lets the chatbot find a passage by meaning and by exact terms. When someone asks a question, the chatbot retrieves only passages that person may open, writes an answer from them and links the section it used. A final check confirms every claim in the draft appears in a retrieved passage. When one does not, the chatbot hands off.
Ovidius built two production chatbots on this pattern, both answering customers rather than employees. Pinkcube's agent answers from a knowledge base built from Pinkcube's own resolved support conversations. Those conversations are filtered before the agent can use them and stored in Supabase pgvector. Gemma, built for Live Platforms' gemstone auction sites, answers from a knowledge base that refreshes weekly from both sites. Our enterprise AI chatbot page describes how it works.
Start with the documents behind the questions your teams answer most often, and only those with someone responsible for keeping them current. Everything else can wait for the second round.
| Source | Owner | Refresh |
|---|---|---|
| HR handbook and policies | HR | On every policy change, with the effective date |
| IT runbooks and how-to pages | IT service desk | When the system changes, and a quarterly review |
| Expense, travel and purchasing rules | Finance | On every change, old versions marked superseded |
| Product and pricing documents | Product or sales operations | With each release |
| Resolved tickets and chats | Support lead | Weekly, filtered for personal data and wrong answers |
| Drafts, meeting notes, personal folders | Nobody | Leave out |
Duplicates and old versions are harder to catch than missing pages, because the chatbot answers from them with a citation. If the 2025 and 2026 travel allowances both sit in the index without dates, the chatbot can retrieve either one. Mark superseded documents, or remove them, before the first ingest. Our article on enterprise knowledge management when AI agents are the readers covers the documentation side in more depth.
A knowledge base chatbot must never show an employee a passage they could not open themselves. The safe way to enforce that is to copy each document's access list into the index and filter every search by who is asking. A restricted passage is then never retrieved for that person in the first place.
A chatbot that respects permissions will still show a file that was shared too widely, as long as the asker can technically open it. Microsoft's own deployment guidance for Copilot, updated in July 2026, says Copilot and agents retrieve data through Microsoft Graph. They respect existing permissions, sharing settings and policies. The same page notes that SharePoint sets sharing to the most permissive option by default. It also lists reports for finding overshared sites before Copilot reaches them. Any knowledge base chatbot that reads SharePoint has the same problem, so audit sharing on the sites you plan to index before the first ingest. Our guide to data governance for AI covers how to run that audit.
When retrieval returns the wrong passage, or none, the model has nothing correct to answer from. Three choices decide what retrieval returns.
Anthropic measured the effect of these choices on its own benchmarks, published in September 2024. It added a short piece of document context to each passage and combined semantic and keyword search. That cut the share of relevant passages missing from the top 20 results by 49%, from 5.7% to 2.9%. Adding a reranking step took the reduction to 67%, down to 1.9%. Your documents will produce different numbers, which is why you test on your own questions. For the wider discipline behind this, read our article on context engineering.
It should say so unless the reply rests on a current passage the asker may open. A fixed rule decides this before the model writes anything, so the chatbot never has to judge its own confidence. Pick a question to see the five outcomes.
The check after the draft matters as much as the rule before it. On IES Limited's daily MI alerts, the database calculates the figures and the model only writes the sentences around them. A verification step then names any figure the model was not handed. Once, the model put two figures into a brief that were not in its input. The brief said so at the top and asked for verification before it went out. The IES case shows the full setup. A knowledge base chatbot needs the same check for policy claims: if a sentence in the draft has no passage behind it, the sentence goes.
Before any customer saw it, Pinkcube's agent went through 76 hostile test scenarios. It scored 4.62 out of 5 on average, with 64 perfect scores. The twelve scenarios that fell short became the business rules the agent runs on. The Pinkcube case shows the worst-scoring scenario and the rule it produced.
Track a small set of numbers from the first week.
The last number feeds the loop in the first diagram. When the HR team sees that leave questions for one country keep going unanswered, it writes that page, and the next ingest closes the gap. For IT and HR desks, our article on internal chatbots for IT and HR service desks covers identity checks and handoffs. Our guide to enterprise chatbot use cases beyond FAQ answers shows what comes after a knowledge base.
A knowledge base chatbot answers employees' questions from your own documents, such as handbooks, runbooks and policies. It retrieves the relevant passages, writes an answer from them and links the source, and it hands off when your documents do not cover the question.
It answers only from the passages it retrieves, and a check removes or blocks any claim with no passage behind it. Retrieval alone does not remove hallucinations. A 2025 Stanford and Yale study found that commercial legal research tools built on their publishers' own databases still hallucinated 17% to 33% of the time. That is why the check and the handoff rule matter.
Yes, if each document's access list is carried into the index and every search is filtered by the identity of the person asking. Audit SharePoint sharing first, because Microsoft's own guidance notes that sharing defaults to the most permissive option.
Only the sources you load first. Give each an owner, mark or remove superseded versions and leave drafts out. The chatbot's unanswered questions then tell each owner which pages to write next.
Copilot suits staff searching and summarizing files they can already open. A custom build fits when answers must follow your rules, cite specific sections, hand off to named owners or combine documents with live system data. If your documents already sit in SharePoint, we can build it on Microsoft Foundry inside your tenant. Our comparison of the best enterprise AI chatbot options covers the trade-offs.
If your documents and permissions need work before any chatbot can answer from them, see how we build an enterprise context layer. You can also book a discovery call and bring the questions your teams ask most.