Why most enterprise RAG pilots stall in week three
The demo works on a handful of clean PDFs, the room is impressed, and then the real document set arrives: scans, merged cells, ten-year-old contracts and three languages. This is the point where most pilots quietly stop.
What happened
The pattern is consistent across deployments we have tracked. Week one is a demo on curated documents. Week two is integration with the customer's identity and permission model. Week three is when someone points the system at the actual document store — and the answer quality collapses.
Why it matters for deployment teams
The failure is almost never the model. It is parsing: tables flattened into unreadable text, scanned pages yielding nothing, and chunks that split a clause from its exceptions. Retrieval then returns confidently wrong context, and the model does what it was told.
What to do about it
Budget the pilot for documents, not models. Parse a representative sample of 200 real files before promising anything, measure context precision on that sample, and make chunking a visible, correctable step rather than a library default.
What to do about it
- Parse 200 real documents before committing to a date
- Make chunking inspectable, not a library default
- Measure faithfulness, not vibes