LLM takes this idea of Bullshit and takes it even further. The model has no concept of truth or facts. It can only pick the most likely word to follow the sequence it has.
A perfect illustration of this for me personally was when I tried early on in the LLM hype cycle (in like 2023? maybe?) playing around with an autocomplete example that said something like “Paris is the capital of France” with a high degree of confidence (which seems impressive until you mess with it) and changing the wording slightly to be a different city…still a high degree of confidence.
I tried telling chatgpt that Versailles was he capital of france to test how it reacts. It corrected me, but what got me was this ending:
For a time […], Versailles was the center of political power in France, but Paris has always remained the official capital.
Let me know if you’re referring to a specific historical period — that might change the context a bit.
Of course, “things might be different in a different period” is a perfectly normal and reasonable thing to say when talking about history, so I imagine it might be common too. If you asked me about the capital of Germany, I’d ask about the period first because that very much changes the answer from “wherever the King happens to be at the moment” to “what Germany?”, “Frankfurt, kinda”, “which part?”, “Berlin”, “which part?” and back to “Berlin”.
I imagine that’s why ChatGPT would add that note: it’s a thing historians are likely to say when asked a question where the answer depends on the exact period. But regardless of whether it is true or not, saying “always” followed by “might change” is a wonderful demonstration that it has no ducking clue why they would say that. If Paris always remained the capital, changing the context won’t fucking change the truth.
LLM takes this idea of Bullshit and takes it even further. The model has no concept of truth or facts. It can only pick the most likely word to follow the sequence it has.
A perfect illustration of this for me personally was when I tried early on in the LLM hype cycle (in like 2023? maybe?) playing around with an autocomplete example that said something like “Paris is the capital of France” with a high degree of confidence (which seems impressive until you mess with it) and changing the wording slightly to be a different city…still a high degree of confidence.
I tried telling chatgpt that Versailles was he capital of france to test how it reacts. It corrected me, but what got me was this ending:
Of course, “things might be different in a different period” is a perfectly normal and reasonable thing to say when talking about history, so I imagine it might be common too. If you asked me about the capital of Germany, I’d ask about the period first because that very much changes the answer from “wherever the King happens to be at the moment” to “what Germany?”, “Frankfurt, kinda”, “which part?”, “Berlin”, “which part?” and back to “Berlin”.
I imagine that’s why ChatGPT would add that note: it’s a thing historians are likely to say when asked a question where the answer depends on the exact period. But regardless of whether it is true or not, saying “always” followed by “might change” is a wonderful demonstration that it has no ducking clue why they would say that. If Paris always remained the capital, changing the context won’t fucking change the truth.