Most enterprise knowledge assistants fail for the same reasons: they show people documents they should not see, quote policies that were replaced months ago, and nobody can say whether they are getting better or worse. Gemini Enterprise provides a strong base — indexed and federated connectors, permissions-aware answers and a customizable RAG agent. Success depends on the work around it: choosing indexed or federated per source, keeping ACLs in sync, parsing documents properly, attaching ownership and lifecycle to content, setting freshness SLOs, deciding when to add a custom ADK agent on RAG Engine or Vector Search, and measuring retrieval and answers with a golden set, online evaluation and Agent Simulation.
Gemini Enterprise on Google Cloud · a five-part series
1. Reference architecture ·
2. Enterprise knowledge bases ·
3. The agentic harness ·
4. Real-time context ·
5. Agentic L1 resolution
Why enterprise knowledge bases disappoint
The pilot usually goes well: a few hundred curated files and a friendly audience. Then scope widens to the real estate — thousands of SharePoint sites, a decade of Confluence, ServiceNow knowledge, shared drives of scanned PDFs — and three problems appear that have nothing to do with the model.
- Permissions. An assistant that can read everything will eventually summarize a salary review, an M&A folder or a disciplinary file for someone who was never meant to see it. One incident like that ends the program.
- Freshness and truth. The travel policy has three versions on three sites. The one the assistant finds first is the 2023 draft. The answer is well written, cited and wrong.
- No measurement. When a business sponsor asks whether answers are good, the honest reply in most programs is “people seem to like it.” Without a golden set, every change is a guess and users find the regressions.
Gemini Enterprise addresses the first problem directly and gives you building blocks for the other two. This post builds on the platform view in part 1.
The standard to design for: every answer is drawn only from content the asking user is allowed to see, from the current version of that content, with a citation they can click — and when the content does not support an answer, the assistant says so.
The architecture
There are two retrieval paths and one access-control path. The platform path uses connectors that index into the Gemini Enterprise search index or federate to the source, answered by the built-in assistant and customizable RAG agent. The custom path is an ingestion pipeline into RAG Engine or Vector Search, read by an ADK agent on Agent Runtime that is registered into the same app. Underneath both: identities and groups from your identity provider, ACLs from each source, and a query-time filter on the person asking.
Two points matter most. The custom path is additive: you register a specialist alongside built-in search, not instead of it. And the ACL path is not optional there: RAG Engine and Vector Search store what you give them, so if chunks do not carry their permitted principals, the custom agent becomes the hole in your permission model.
Components mapped to Google Cloud
| Layer | Google Cloud service | What it does in the knowledge base |
|---|---|---|
| Experience | Gemini Enterprise app | Built-in search and assistant, customizable RAG agent, Gemini Notebook Enterprise, Agent Gallery |
| Connect | Prebuilt connectors | SharePoint, Confluence, Jira, ServiceNow, Google Workspace and more; each source either indexed into the search index or federated at query time |
| Access control | Permissions-aware search, Cloud IAM, your identity provider | Access-controlled search results and generative answers; identities and groups resolved per user |
| Custom ingestion | Cloud Storage, Pub/Sub, Eventarc, Cloud Run, Workflows | Change detection, parsing, chunking, metadata enrichment and upserts for content outside the connector path |
| Custom retrieval | RAG Engine, Vector Search | Managed corpus and retrieval for a specialist agent; vector index with metadata filtering at larger scale |
| Custom agent | ADK on Agent Runtime, registered in the app | Domain-specific retrieval, reranking, answer policy and refusal rules; admins choose which users can use it |
| Metadata & ownership | Dataplex, BigQuery | Content inventory, owners, effective and review dates, duplicate clusters, freshness metrics |
| Protection | VPC Service Controls, CMEK with Cloud KMS, Model Armor, Sensitive Data Protection | Perimeter around data and models, customer-managed keys, prompt-injection screening, PII discovery and masking |
| Quality | Agent evaluation, online evaluation, Agent Simulation, Observability | Golden-set regression, sampled production scoring, synthetic multi-turn users, traces of every retrieval and answer |
Indexed or federated: choosing per source
Gemini Enterprise connectors work in one of two ways. An indexed connector copies content — and the permissions attached to it — into Gemini Enterprise’s search index and keeps it in sync. A federated connector sends the query to the source system at question time and uses what comes back, without copying anything. Decide per source; most estates use both.
| Consideration | Favors indexed | Favors federated |
|---|---|---|
| Answer quality | You need ranking across sources, semantic retrieval over full text, and consistent snippets | The source has strong native search for its own content type |
| Data residency and copying | A governed copy inside your Google Cloud boundary is acceptable | Policy or contract forbids copying the content out of the source |
| Freshness | Minutes-level lag from sync is acceptable | Content changes by the minute and must be exact (ticket status, live records) |
| Volume and churn | Large, relatively stable corpora: policies, manuals, wikis | Enormous or very high-churn data where a full copy is costly to maintain |
| Permissions | Source ACLs can be read and synced reliably | Source permissions are complex or computed, and only the source can evaluate them correctly |
The common pattern: index document estates (SharePoint, Confluence, Drive, knowledge articles) and federate transactional systems where the question is about a live record. Federated results are only as good as the source’s own search, so test them against your golden set; if it is weak and copying is allowed, index instead.
A useful default: index content people read, federate records people look up. Revisit any source whose federated answers score badly in evaluation.
Permission-aware retrieval and ACL sync
Permissions-aware access means results and generative answers draw only on what the asking user could open in the source. The guarantee holds while three things stay true: indexed ACLs match the source, source identities map to the identity that signs in, and groups resolve correctly. Each can break.
What breaks
- Stale ACLs. Someone is removed from a confidential site at 9:00. If permissions are synced on a schedule, there is a window in which the index still lists them. Sync permission changes faster than content; for the most sensitive sources consider federation or exclusion.
- Group resolution. Most enterprise ACLs are on groups, often nested, sometimes defined in the source system rather than the directory (SharePoint site groups, Confluence space permissions, Jira project roles). If a source-local group cannot be expanded, documents become invisible to the right people or mapped too broadly. Inventory where groups live before connecting.
- Identity mapping.
jsmith@corp.comin the directory isjohn.smithin an old wiki. Without a reliable mapping, documents silently disappear for the people who own them. - External and guest users. Contractors and partner guests appear in source ACLs with identities your workforce directory may not know. Decide explicitly whether guests use the assistant, and test documents shared only with external accounts.
- “Everyone” links. Years of “anyone in the organization with the link” sharing mean the assistant will find documents nobody realized were broadly readable. The assistant makes old oversharing discoverable; review high-risk sites before connecting them.
The custom path carries its own ACLs
For content you ingest yourself into RAG Engine or Vector Search, store the allowed principals (users and expanded groups) as metadata on every chunk and filter on them inside the retrieval call, using the end user’s identity passed through from the Gemini Enterprise app — never the agent’s service identity. Filter before ranking: a forbidden document must never enter the model’s context. The agent harness in part 3 covers how Agent Identity and OAuth authorizations carry the user through.
Test permissions like a security control, because it is one. Keep “canary” documents restricted to test users and query them daily as allowed and disallowed users. A leaked canary is a severity-one incident.
Ingestion: parsing, chunking and metadata
For sources on the connector path, Gemini Enterprise handles extraction. For everything on the custom path — scanned archives, export dumps, documents with complex layout, content from systems without a connector — ingestion quality decides answer quality.
Parsing: layout, tables and scans
Plain text extraction flattens a document: two-column layouts interleave, headers and footers repeat into every paragraph, and tables become a stream of numbers with no column meaning. Use a layout-aware parser that preserves heading hierarchy, reading order and table structure. Convert tables to a structured form (Markdown or HTML rows with header cells repeated) so that “the deductible for tier 3” can be found and quoted with its row and column. For scanned PDFs, run OCR, keep the per-page confidence, and route low-confidence pages to review rather than indexing noise. Record the parser version so improvements can be re-applied selectively.
Chunking
Fixed-size chunking is the most common source of “the answer was in the document but the assistant missed it.” Chunk on the document’s own structure: section and sub-section boundaries, with the heading path (“Travel Policy › International › Per diem”) prepended to each chunk so it carries its context. Keep tables, numbered procedures and definitions intact even if they exceed your target size. Keep a pointer to the parent section so the agent can expand context when a chunk is ambiguous.
Metadata that earns its keep
Every chunk carries source URL, document ID and version, heading path, owner, document type, effective and review dates, jurisdiction, allowed principals, parser version and content hash. Metadata is what lets you filter, boost the policy over a slide quoting it, explain (“effective 1 March”) and retire.
Built-in search or a custom RAG agent
Built-in search and the customizable RAG agent are the right first answer for most questions. Add a custom ADK agent on RAG Engine or Vector Search, hosted on Agent Runtime, when one of these signals appears — and keep the built-in path for everything else.
| Signal | Built-in is enough | Add a custom RAG agent |
|---|---|---|
| Sources | Covered by prebuilt connectors | Scanned archives, exports or systems with no connector |
| Documents | Mostly text pages and office documents | Complex tables, forms or layouts that need specialist parsing |
| Answer policy | General cited answers are acceptable | Strict rules: answer only from the current effective version, always quote the clause, refuse outside scope |
| Retrieval | Default ranking performs well on the golden set | Domain filters, reranking or multi-step retrieval are needed to pass evaluation |
| Workflow | Question in, answer out | The answer feeds a step: a draft reply, a checklist, a referral to a human |
| Audit | Standard logging is sufficient | Regulators need the exact chunks, versions and rules behind every answer |
The custom agent is registered into the Gemini Enterprise app (Agents > Add agent, or the REST API), and admins choose which users can use it. Employees still start in one place: a general assistant, plus a few specialists where the answer policy justifies one.
Grounding, citations and refusing when unsupported
A grounded answer is one where every claim can be traced to a retrieved passage the user can see. Three rules make that real rather than aspirational.
Cite at the claim, not at the bottom
Citations attached to individual sentences, linking to the source document and, where possible, the section, let a reader verify the one sentence they care about. Show the document’s effective or last-modified date next to the citation.
Refuse when retrieval does not support an answer
If the retrieved passages do not answer the question, the correct response is “I could not find this in the sources available to you,” plus where to go next (the policy owner, an HR case, a team channel). In a custom agent, enforce this with a support check: before returning, verify that each claim is entailed by a cited passage and drop or flag what is not.
Do not let the model fill gaps from general knowledge
The model knows what a typical parental-leave policy says, not yours. Instruct and evaluate for sources-only behavior, and when sources conflict, show both citations rather than choosing silently.
Freshness SLOs and content lifecycle
A knowledge base has two freshness numbers, and both need a target, a measurement and an alert. Content freshness is the time from a change in the source to that change being retrievable. Permission freshness is the time from an access change in the source or directory to the index reflecting it. For example, a team might target a few hours for wikis, an hour for policies, and tighter for permissions. Federated sources are fresh by construction.
Re-index strategy
- Incremental by default. Use change events or modified-since crawls; skip unchanged files by content hash.
- Fast lane for deletions and ACL changes. A deleted or restricted document must leave the index faster than a new document enters it.
- Targeted re-processing. When the parser, chunker or embedding model changes, rebuild into a new index version, evaluate it against the golden set, and switch over only when it is at least as good — blue-green for indexes.
- Reconcile and measure. Compare index and source inventories on a schedule to catch missed events, and publish freshness percentiles per source in Cloud Monitoring.
Ownership, staleness and duplicates
A fresh pipeline does not help if the content is stale. Every document needs an owner and a review date. Build a content inventory in BigQuery from captured metadata and send owners three weekly reports: past review date, no active owner, and duplicate clusters. Owners confirm or retire; superseded versions are excluded; one canonical version wins. Documents often cited in answers marked wrong go first.
Retirement is a feature. Excluding an obsolete policy from retrieval improves answers more reliably than any prompt change.
Five use cases, and what each one demands
1. Employee policy and HR helpdesk
Leave, benefits, travel and expenses. The hard parts are jurisdiction and versions: Ontario differs from Texas, and the policy changed in January. Tag policies by country, employee type and effective date, filter on the employee’s attributes, and refuse personal-case questions (“how much leave do I have left?”) unless a governed tool answers them from the HR system.
2. Engineering knowledge
Runbooks, decision records, design docs, Jira and Confluence. Engineers expect the current runbook, not the one from two migrations ago. Index wikis, federate live ticket status, boost documents linked from the service catalog, and pair the assistant with the real-time context in part 4 when a question is about what is happening now rather than how things work.
3. Customer-service agent assist
The assistant suggests cited answers to a human agent from knowledge articles and product documentation. Measure acceptance rate alongside groundedness, and feed rejected suggestions into content review. The same knowledge base later powers the resolution agents described in part 5.
4. Sales enablement
Product sheets, pricing guidance, competitive notes and security questionnaires. The risks are stale pricing and unapproved claims. Restrict customer-facing drafts to approved collections, show effective dates, and keep internal competitive notes behind group ACLs.
5. Regulated knowledge: insurance underwriting guidelines
Underwriting guidelines are versioned by line of business, state and effective date, full of tables (rating factors, eligibility limits, referral thresholds) and frequently amended by bulletin. This is where a custom ADK agent earns its place: specialist table-aware parsing, strict filtering to the guideline in force for the risk’s state and effective date, answers that quote the clause and its version, and refusal plus referral to a senior underwriter when the guideline is silent. Every answer is logged with exact chunks and versions for audit.
Walkthrough: one question, end to end
An underwriter in a commercial lines team asks the underwriting guidelines agent in the Gemini Enterprise app: “For a restaurant risk in Florida, what is the maximum TIV we can write without referral?”
Identity and routing
The underwriter is signed in to Gemini Enterprise. The guidelines agent is registered in the app and enabled for the underwriting group only. The request reaches the ADK agent on Agent Runtime carrying the user’s identity.
Screening and filtered retrieval
Model Armor screens the prompt. The agent extracts the facets (line: commercial property, class: restaurant, state: Florida, date: today) and queries the RAG Engine corpus with metadata filters for document type = guideline, state in {FL, all}, effective date ≤ today and not superseded, and allowed principals containing the user or their groups. Top passages include the Florida table of referral thresholds and a bulletin amending it last month.
Screening retrieved content
Retrieved passages are screened by Model Armor for embedded instructions before they enter the model’s context.
Grounded answer with a conflict surfaced
The agent answers with the amended threshold from the bulletin, cites the bulletin clause and its effective date, notes that it supersedes the figure in the base guideline table (also cited), and states that risks above the threshold require referral. A support check confirms each sentence is backed by a cited passage.
Record and learn
The trace — query, filters, chunk IDs and versions, screening results, answer — goes to Cloud Logging. Online evaluation samples the turn and scores groundedness. If the underwriter marks the answer unhelpful, it enters the triage queue.
Security, identity and governance
- VPC Service Controls. Put the corpus, indexes, ingestion buckets and agent runtime inside a service perimeter to block exfiltration even with valid credentials.
- CMEK. Customer-managed keys in Cloud KMS for buckets, indexes and logs where policy requires it.
- Model Armor against prompt injection inside documents. In a knowledge base, the most likely injection does not come from the user; it comes from a document. A wiki page saying “ignore your instructions” is retrieved like any other content. Screen retrieved passages as well as prompts, treat retrieved text as data in the agent’s instructions, and do not give knowledge agents tools with side effects they do not need.
- Sensitive Data Protection (DLP). Scan for personal and regulated data at ingestion and mask or exclude it; run discovery on connector sources too, because an assistant makes forgotten spreadsheets findable.
- Least privilege for agents. Each agent has its own identity with read-only access to its corpus; retrieval filters use the end user’s identity. Agent Registry and Agent Gateway, covered in part 3, keep the inventory and the policy enforcement point in one place.
- Audit. Log query, user, filters, retrieved chunk IDs and versions, and the answer. Restrict who can read logs containing question text.
Evaluation: making quality measurable
A knowledge base is measurable when you can answer two questions with numbers: did we retrieve the right passages, and did we answer correctly from them? Separating the two matters, because the fixes are different — a retrieval miss is fixed in content, parsing, chunking or metadata; an answer error is fixed in instructions, answer policy or model choice.
The golden question set
Collect a few hundred real questions per domain from tickets, search logs and experts. For each, record expected sources, the key facts the answer must contain, and the persona asking. Include three special slices: must-refuse questions the sources do not answer, conflict questions where an old and a new version both exist, and permission canaries asked as users who must not see the answer. Content owners sign off on reference answers.
Metrics that separate retrieval from answering
| Metric | What it tells you |
|---|---|
| Retrieval recall@k | Was at least one expected source in the top k passages given to the model? |
| Rank of first correct source | How high the right passage ranks; drives answer quality and context cost |
| Answer correctness | Does the answer contain the reference facts, judged by an autorater calibrated against human grades? |
| Groundedness | Is every claim supported by a retrieved passage? |
| Citation accuracy | Does each citation actually support the sentence it is attached to? |
| Correct refusal rate | On must-refuse questions, did it decline? On answerable ones, did it avoid refusing? |
| Permission leakage | Canary content in answers for disallowed users; the target is zero |
| Freshness | Source-change-to-retrievable percentiles per source, against SLO |
Offline, simulated and online
Run the golden set through agent evaluation, with multi-turn autoraters, on every change to connectors, parsing, chunking, prompts or models; block any regression. Agent Simulation adds synthetic users who follow up, rephrase or push back on a refusal — failures that appear on turn three. In production, online evaluation scores sampled turns and thumbs-down feedback goes to triage. Calibrate autoraters against human grades before trusting them.
A 90-day rollout plan
Days 1–30: one domain, measured
Pick one domain with clear owners and real demand — HR policy is a common choice. Inventory its sources, owners and groups; run an oversharing review; decide indexed or federated per source; connect them. Build the golden set with the domain team, including canaries and must-refuse questions. Baseline the built-in assistant against it. Set content and permission freshness SLOs and start measuring.
Days 31–60: close the gaps
Fix what the baseline exposed: retire duplicates, assign missing owners, add metadata, tune the customizable RAG agent. If evaluation shows the built-in path cannot meet the bar for a sub-domain, build the custom ingestion pipeline and an ADK agent for it, with ACL metadata and Model Armor screening from day one. Wire up VPC Service Controls, CMEK and DLP scanning. Launch to a pilot group with feedback capture and online evaluation on.
Days 61–90: operate and expand
Put the release gate into the deployment pipeline. Start weekly content-lifecycle reports to owners. Review production failures by cause and grow the golden set. Go live for the first domain and start the second with the same playbook, reusing the pipeline, evaluation harness and controls.
Failure modes we plan for
1. Connecting everything on day one. Thousands of sites with unknown owners and years of oversharing become instantly searchable. Connect by domain, after an oversharing review, with owners named.
2. A custom agent that ignores ACLs. The pipeline loads documents into a corpus, the agent queries it with its own service identity, and every user can now read everything in it. Chunks carry principals; retrieval filters on the end user.
3. Deletions that never propagate. A recalled document stays retrievable because the pipeline only handles creates and updates. Deletions and ACL changes take the fast lane, and reconciliation catches what events miss.
4. Tables flattened into noise. Rate tables and eligibility grids lose their structure, so the assistant quotes the wrong row with confidence. Layout-aware parsing, tables kept whole, and table questions in the golden set.
5. Three versions of the truth. Old and new policies both rank, and the assistant picks one. Effective dates, supersession metadata and canonical versions; surface conflicts instead of choosing silently.
6. Instructions hidden in documents. A retrieved page tells the model to do something else. Screen retrieved content with Model Armor, treat it as data, and keep knowledge agents free of unneeded tools.
Readiness checklist
| Area | Ready when |
|---|---|
| Scope | First domain chosen, with named content owners and a business sponsor |
| Connectors | Each source assigned indexed or federated, with the reason recorded |
| Permissions | Group sources inventoried, identity mapping tested, guests decided, oversharing reviewed |
| Ingestion | Layout-aware parsing, table handling, OCR confidence and quarantine in place for the custom path |
| Metadata | Owner, effective and review dates, type, ACL principals and version on every chunk |
| Freshness | Content and permission SLOs defined per source, measured and alerted on |
| Lifecycle | Weekly stale, ownerless and duplicate reports reach owners; supersession is honored in the index |
| Answer policy | Claim-level citations, refusal when unsupported, conflicts surfaced |
| Security | VPC Service Controls perimeter, CMEK where required, Model Armor on prompts and retrieved content, DLP scanning |
| Evaluation | Golden set with must-refuse, conflict and canary slices; release gate; online evaluation and Agent Simulation in use |
| Operations | Traces for every turn; failures triaged by cause and added to the golden set |
The bottom line
Gemini Enterprise removes much of the plumbing that used to consume knowledge-base projects: connectors, a permissions-aware index, cited answers and an app employees already use, with a clean way to add specialist ADK agents on RAG Engine or Vector Search when a domain needs one. What the platform cannot do for you is decide which sources to index or federate, keep your groups and owners in order, retire the policy that was replaced last spring, or tell you whether this week’s change made answers better. Treat permissions as a security control with canaries, freshness as an SLO, content as something with owners and an end date, and quality as a number with a release gate. That trust is what lets the knowledge base become the foundation for the agents in the rest of this series.
At VerticalServe we have spent seven years building data and AI platforms on Google Cloud and run production agentic systems in insurance. Our InsightAsk and InsightRAG accelerators package the patterns described here and run alongside Gemini Enterprise, not in place of it.