Retrieval adds documents, not guaranteed correctness
Giving a model access to your documents makes its answers more relevant. It does not make them true.
The model still generates a plausible continuation, now shaped by whatever was retrieved. If the retrieval brought back an outdated policy, a draft, or the wrong document entirely, the answer will be confidently based on it. Retrieval improves grounding; it does not add verification, and the two get conflated constantly in product descriptions.
More on AI and LLMs
- An LLM predicts plausible continuations, not verified truthIt only checks the shape
- Context is temporary working material, not permanent knowledgeThe board gets wiped
- Temperature changes variation, not factualityThe dial only sets the spread
- System prompts are instructions, not a security boundaryA sign, with no fence
- LLM output is untrusted input downstreamIt comes in round the back
- Fine-tuning changes behaviour, not every limitationNew type, same roller