/ Context
Enterprise answers sit across a very large document estate — a 500GB+ corpus that had to be embedded, retrieved over and served behind an agent. Retrieval quality, not model choice, was the thing that determined whether answers could be relied on, so the platform was built around it.
/ Role & decision scope
- Architected and delivered the platform end to end: ingestion, embedding, retrieval and agent architecture.
- Selected and combined the retrieval techniques: hybrid search, reranking and vector retrieval.
- Chose a ReAct agent architecture over the retrieval layer.
- Delivered the platform, not only the design.
/ Constraints
- A 500GB+ enterprise corpus to embed and ingest.
- Answers have to be grounded in retrieved enterprise content rather than model recall.
- Response latency matters — the system is used interactively.
- Retrieval accuracy had to be measurable, not asserted.
/ Architecture & key decisions
- D01
Hybrid search combined with reranking
Retrieval combines hybrid search, reranking and vector retrieval rather than relying on a single retrieval strategy, so candidate generation and final selection are separate concerns.
- D02
ReAct agent architecture
A ReAct architecture sits over the retrieval layer, letting the agent reason across steps instead of answering from a single retrieval pass.
- D03
A 500GB+ ingestion and embedding pipeline as platform infrastructure
Ingestion and embedding at this scale were built as a pipeline in their own right, so the corpus is a maintained asset of the platform rather than a one-off load.
- D04
Accuracy tracked as a measured number
Design focus: retrieval quality is treated as the product. Chatbot accuracy was measured and improved to 95%, which is what makes retrieval work reviewable rather than subjective.
/ Trade-offs
- Retrieval quality versus response latency
- Both were treated as targets rather than one being sacrificed: accuracy improved to 95% and response latency was reduced.
- Single retrieval strategy versus a combined one
- Accepted the additional moving parts of hybrid search plus reranking on top of vector retrieval, because retrieval quality was the deciding factor for trust.
- Agent flexibility versus predictability
- A ReAct architecture was chosen for multi-step reasoning over retrieved evidence, keeping the retrieval layer — not the model — as the source of answers.
/ Evidence & outcomes
- Measured
- Measured chatbot accuracy improved to 95%
- Measured
- Response latency reduced
- Artifact
- 500GB+ enterprise embedding and ingestion pipeline
- Artifact
- Hybrid search, reranking and vector retrieval layer behind a ReAct agent architecture
- Decision
- Retrieval quality treated as the measured product of the platform