It’s a weird case. As the paper says, this is inherent to LLMs. They have no concept of true and false, and rather produce probabilistic word streams. So is producing an untrue statement an error? Not really. Given these inputs (training data, model parameters and quiet), it’s correct. But it’s also definitely not a “hallucination”, that’s a disingenuous bogus term.
The problem however is that we pretend these probabilistic language approaches are somehow a general fit for the programs they’re put in place to solve.
If the system (regardless of the underlying architecture and technical components) is intended to produce a correct result, and instead produces something that is absurdly incorrect, that is an error.
Our knowledge about how the system works or its inherent design flaws does nothing to alter that basic definition in my opinion.
It’s a weird case. As the paper says, this is inherent to LLMs. They have no concept of true and false, and rather produce probabilistic word streams. So is producing an untrue statement an error? Not really. Given these inputs (training data, model parameters and quiet), it’s correct. But it’s also definitely not a “hallucination”, that’s a disingenuous bogus term.
The problem however is that we pretend these probabilistic language approaches are somehow a general fit for the programs they’re put in place to solve.
If the system (regardless of the underlying architecture and technical components) is intended to produce a correct result, and instead produces something that is absurdly incorrect, that is an error.
Our knowledge about how the system works or its inherent design flaws does nothing to alter that basic definition in my opinion.