The first version of our FDE assistant was impressive and useless. Ask it where retry logic lived and it would answer immediately, fluently, and about thirty percent of the time, wrongly. Wrongly in the specific way that costs you an hour: a real-sounding path, a plausible function name, a confident explanation of code that did not exist.
An engineer who is wrong thirty percent of the time and sounds certain is not a junior colleague. They are a liability, because verifying their claims costs more than doing the work.
So we made a rule: no factual claim about a codebase without a citation to the file and line it came from. If the retrieval layer cannot find supporting code, the assistant says it cannot find it. Not a hedge, not a guess with a caveat — an explicit "I don't have this".
Three things happened.
Answer volume dropped. Roughly one in six questions started returning "not found" instead of prose. Uncomfortable to watch, and correct.
Trust went up sharply. Engineers stopped double-checking every answer, because they could check any answer in two seconds by clicking the citation. Verifiability, not accuracy, is what makes a tool usable.
The product's job changed. Once answers were checkable, people started asking harder questions — not "where is this function" but "what breaks if I change this contract". Those questions have architectural answers that span twenty files, and they are where the actual time savings live.
The uncomfortable lesson is that fluency was doing us harm. A model that writes well is very good at making a wrong answer feel like a right one, and no amount of prompt engineering fixes that. The fix was structural: make the system unable to assert what it cannot show.
If you build on top of a model in a domain where being wrong is expensive, the design question is not "how do we make it more accurate". It is "how does the user cheaply tell whether this particular answer is true".