Stage 02 of 6 · The Closed Loop
Normalize — Parse, enrich, embed and tag — turn signal into searchable structure.
Stage 02 of Implement Agentic's six-stage closed loop. Below: process, technologies, ecosystem partners, build-vs-buy guidance, project lifecycle and the benefits clients see.
Process
- Step 01
Parse documents with vision-aware extractors that preserve tables, headings and citations.
- Step 02
Chunk semantically (~500 tokens, 100-token overlap) with by-title and by-page strategies for tables/code.
- Step 03
Add Anthropic-style contextual prefixes (50-token 'what document/section is this').
- Step 04
Generate embeddings with the model that matches your query distribution (text-embedding-3-large, Cohere v3, Voyage).
- Step 05
Stamp every chunk with document, section, author, ACL, classification and date metadata.
Technologies
The 2026 production stack for this stage. Implement Agentic is deliberately model- and vendor-neutral.
Document parsers
- Unstructured.io (broadest format coverage)
- LlamaParse (best for tables)
- Reducto (regulated, on-prem, citation-grade)
- Docling (ESG/finance hierarchical docs)
- Azure Document Intelligence (invoices, IDs)
Embedding models
- OpenAI text-embedding-3-large
- Cohere Embed v3
- Voyage
- BGE / e5 (open weight)
Enrichment
- spaCy / Presidio (PII)
- GLiNER (entity extraction)
- Custom classifiers
Ecosystem partners
Unstructured.io
General document ingestion + chunking pipeline
LlamaParse
Tables and contracts/SOWs
Reducto
Regulated, on-prem, citation-grade extraction
See the full Ecosystem partners page for our tool catalog.
Build-vs-buy decisions
Buy when…
Buy parsing as a service (LlamaParse, Unstructured Platform, Reducto) — accuracy and format coverage compound quickly and are not a differentiator.
Build when…
Build the chunking, ACL-tagging and contextual-prefix logic — these encode your business semantics and must stay portable.
Implement Agentic's recommendation
Hybrid: Unstructured for general docs, LlamaParse/Reducto for high-stakes tables and contracts, custom enrichment for ACL and entity tags.
Project lifecycle
| Phase | Timing | Outcome |
|---|---|---|
| Schema design | Week 1 | Chunk schema, metadata fields and ACL model agreed. |
| Pipeline build | Weeks 2–3 | End-to-end parser → chunker → embedder → index pipeline. |
| QA pass | Week 4 | Sampled extractions reviewed by SMEs; reranker calibrated. |
| Operate | Ongoing | Re-embedding triggered on model upgrades only; embeddings cached. |
Benefits
- Recall@10 of ~91% with hybrid BM25 + dense + RRF + cross-encoder reranking.
- Costs amortized via embedding cache — no re-embedding on every change.
- ACL tagging at chunk time means retrieval-time filtering is a single SQL predicate.
Failure modes we prevent
- Naive fixed-size chunks that split tables in half.
- Lost ACLs between parser and index, breaking permission enforcement downstream.
- Re-embedding the entire corpus on every minor change.