An AI that lacks intelligence is only ever going to do what mathematics does. If predictive mathematics were accurate, we’d literally be able to see the future and we wouldn’t call it “predictive”.
“Hallucinations” has always been an inaccurate term, but I think it was picked to imply intelligence was there when it never was.
The best use case for this is on social media to use it to manipulate public opinion. That’s why all social media companies are heavily invested in it.
Thus, bullshit, in contrast to mere nonsense, is something that implies but does not contain adequate meaning or truth.
We argue that an important adjutant of pseudo-profound bullshit is vagueness which, combined with a generally charitable attitude toward ambiguity, may be exacerbated by the nature of recent media.
The concern for “profundity” reveals an important defining characteristic of bullshit (in general): that it attempts to impress rather than to inform; to be engaging rather than instructive.
I think even bullshitting isn’t a good term for it because to me it implies intent.
It’s just a text predictor that can predict text well enough to be conversational and trick people interacting with it enough to pass the Turing test (which IMO was never really a good test of intelligence, though maybe shines a spotlight on how poorly “intelligence” is defined in that context, because despite not being a good test, it might still be one of the best I’ve heard of).
All of its “knowledge” is in the form of probabilities that various words go together, given what words preceded them. It has no sense of true, false, or paradox.
The problem with LLM hallucinations is not a general limitation of mathematics or linear algebra.
The problem is that the LLMs fall into bullshit, in the sense of On Bullshit. The deal is that both truthtellers and liars care about what the real truth is, but bullshit ters simply don’t care at all whether they’re telling the truth. The LLMs end up spouting bullshit, because bullshit is designed to be a pretty good solution to the natural language problem; and there’s already a good amount of bullshit in the LLM training data.
LLM proponents believed that if you put enough compute power at the problem of predicting the next token, then the model will be forced to learn logic and math and everything else to keep optimizing that next token. The existence of bullshit in natural language prevents this from happening, because the bullshit maximizes the objective function at least as well as real content.
LLM takes this idea of Bullshit and takes it even further. The model has no concept of truth or facts. It can only pick the most likely word to follow the sequence it has.
A perfect illustration of this for me personally was when I tried early on in the LLM hype cycle (in like 2023? maybe?) playing around with an autocomplete example that said something like “Paris is the capital of France” with a high degree of confidence (which seems impressive until you mess with it) and changing the wording slightly to be a different city…still a high degree of confidence.
I tried telling chatgpt that Versailles was he capital of france to test how it reacts. It corrected me, but what got me was this ending:
For a time […], Versailles was the center of political power in France, but Paris has always remained the official capital.
Let me know if you’re referring to a specific historical period — that might change the context a bit.
Of course, “things might be different in a different period” is a perfectly normal and reasonable thing to say when talking about history, so I imagine it might be common too. If you asked me about the capital of Germany, I’d ask about the period first because that very much changes the answer from “wherever the King happens to be at the moment” to “what Germany?”, “Frankfurt, kinda”, “which part?”, “Berlin”, “which part?” and back to “Berlin”.
I imagine that’s why ChatGPT would add that note: it’s a thing historians are likely to say when asked a question where the answer depends on the exact period. But regardless of whether it is true or not, saying “always” followed by “might change” is a wonderful demonstration that it has no ducking clue why they would say that. If Paris always remained the capital, changing the context won’t fucking change the truth.
It’s a weird case. As the paper says, this is inherent to LLMs. They have no concept of true and false, and rather produce probabilistic word streams. So is producing an untrue statement an error? Not really. Given these inputs (training data, model parameters and quiet), it’s correct. But it’s also definitely not a “hallucination”, that’s a disingenuous bogus term.
The problem however is that we pretend these probabilistic language approaches are somehow a general fit for the programs they’re put in place to solve.
If the system (regardless of the underlying architecture and technical components) is intended to produce a correct result, and instead produces something that is absurdly incorrect, that is an error.
Our knowledge about how the system works or its inherent design flaws does nothing to alter that basic definition in my opinion.
but I think it was picked to imply intelligence was there when it never was.
Bingo bango. This is why the humanizing words used with these algorithms are so insidious. They have all been adopted and promoted to subtly suggest and enforce the idea that there is intelligence, or even humanity, where there is none. It’s what sells the hype and inflates the bubble.
Yep. Its one of those “um actually” things that at surface can make you seem annoying, but unfortunately the nuance is really important.
In order to hallucinate it’d need to be capable of proper thought first.
In the same way people ask of their software “why doesn’t it just work?!”
Well… It actually DOES work. Its doing exactly as it’s been programmed to do.
Whether the issue is because the dev didnt think of an angle you use it on, the QA didnt test it enough, or you yourself have a weird expectation, etc, it is doing exactly what it is only capable of doing in the situation that you see as “it isnt working right”.
Its then on you, the human, to recognize that and proceed.
This dissonance even happens from human to human conversation. “Oh i thought you meant this.”
If you go to an agriculturist and start asking them about the culture of another country, they’d probably stop you to point out the issue. They could also just start giving you agriculture info and leave you confused. The nuance is important and what lets our biological brain computer figure it out where the metal brain needs to be specifically told to make sure they meant land agriculture.
An AI that lacks intelligence is only ever going to do what mathematics does. If predictive mathematics were accurate, we’d literally be able to see the future and we wouldn’t call it “predictive”.
“Hallucinations” has always been an inaccurate term, but I think it was picked to imply intelligence was there when it never was.
The precise, scientific term is “bullshitting”.
The best use case for this is on social media to use it to manipulate public opinion. That’s why all social media companies are heavily invested in it.
Indeed: https://www.cambridge.org/core/journals/judgment-and-decision-making/article/on-the-reception-and-detection-of-pseudoprofound-bullshit/0D3C87BCC238BCA38BC55E395BDC9999
I think even bullshitting isn’t a good term for it because to me it implies intent.
It’s just a text predictor that can predict text well enough to be conversational and trick people interacting with it enough to pass the Turing test (which IMO was never really a good test of intelligence, though maybe shines a spotlight on how poorly “intelligence” is defined in that context, because despite not being a good test, it might still be one of the best I’ve heard of).
All of its “knowledge” is in the form of probabilities that various words go together, given what words preceded them. It has no sense of true, false, or paradox.
Misinformation farms ADORE it
Predictive mathematics is highly accurate and quite useful at predicting the future already for many types of problems.
As one example: we can use math models to predict where the planets in the solar system will be.
The problem with LLM hallucinations is not a general limitation of mathematics or linear algebra.
The problem is that the LLMs fall into bullshit, in the sense of On Bullshit. The deal is that both truthtellers and liars care about what the real truth is, but bullshit ters simply don’t care at all whether they’re telling the truth. The LLMs end up spouting bullshit, because bullshit is designed to be a pretty good solution to the natural language problem; and there’s already a good amount of bullshit in the LLM training data.
LLM proponents believed that if you put enough compute power at the problem of predicting the next token, then the model will be forced to learn logic and math and everything else to keep optimizing that next token. The existence of bullshit in natural language prevents this from happening, because the bullshit maximizes the objective function at least as well as real content.
LLM takes this idea of Bullshit and takes it even further. The model has no concept of truth or facts. It can only pick the most likely word to follow the sequence it has.
A perfect illustration of this for me personally was when I tried early on in the LLM hype cycle (in like 2023? maybe?) playing around with an autocomplete example that said something like “Paris is the capital of France” with a high degree of confidence (which seems impressive until you mess with it) and changing the wording slightly to be a different city…still a high degree of confidence.
I tried telling chatgpt that Versailles was he capital of france to test how it reacts. It corrected me, but what got me was this ending:
Of course, “things might be different in a different period” is a perfectly normal and reasonable thing to say when talking about history, so I imagine it might be common too. If you asked me about the capital of Germany, I’d ask about the period first because that very much changes the answer from “wherever the King happens to be at the moment” to “what Germany?”, “Frankfurt, kinda”, “which part?”, “Berlin”, “which part?” and back to “Berlin”.
I imagine that’s why ChatGPT would add that note: it’s a thing historians are likely to say when asked a question where the answer depends on the exact period. But regardless of whether it is true or not, saying “always” followed by “might change” is a wonderful demonstration that it has no ducking clue why they would say that. If Paris always remained the capital, changing the context won’t fucking change the truth.
Hallucinations are investor / booster speak for errors.
It’s a weird case. As the paper says, this is inherent to LLMs. They have no concept of true and false, and rather produce probabilistic word streams. So is producing an untrue statement an error? Not really. Given these inputs (training data, model parameters and quiet), it’s correct. But it’s also definitely not a “hallucination”, that’s a disingenuous bogus term.
The problem however is that we pretend these probabilistic language approaches are somehow a general fit for the programs they’re put in place to solve.
If the system (regardless of the underlying architecture and technical components) is intended to produce a correct result, and instead produces something that is absurdly incorrect, that is an error.
Our knowledge about how the system works or its inherent design flaws does nothing to alter that basic definition in my opinion.
Yeah it’s a pretty good hand wavey term for a real issue
Bingo bango. This is why the humanizing words used with these algorithms are so insidious. They have all been adopted and promoted to subtly suggest and enforce the idea that there is intelligence, or even humanity, where there is none. It’s what sells the hype and inflates the bubble.
To be fair, some AI is just people in a data center manually generating responses.
Yep. Its one of those “um actually” things that at surface can make you seem annoying, but unfortunately the nuance is really important.
In order to hallucinate it’d need to be capable of proper thought first.
In the same way people ask of their software “why doesn’t it just work?!” Well… It actually DOES work. Its doing exactly as it’s been programmed to do.
Whether the issue is because the dev didnt think of an angle you use it on, the QA didnt test it enough, or you yourself have a weird expectation, etc, it is doing exactly what it is only capable of doing in the situation that you see as “it isnt working right”.
Its then on you, the human, to recognize that and proceed.
This dissonance even happens from human to human conversation. “Oh i thought you meant this.”
If you go to an agriculturist and start asking them about the culture of another country, they’d probably stop you to point out the issue. They could also just start giving you agriculture info and leave you confused. The nuance is important and what lets our biological brain computer figure it out where the metal brain needs to be specifically told to make sure they meant land agriculture.
ah yes, psychohistory. math predicting the future is the initial idea that gets asimov’s “foundation” rolling.