I think Americans are primed to feel very enthusiastic about anything that tells them things they already believe, even moreso if it sounds very confident and organized. And people are already prone to woowoo stuff.

I don’t think you’d break the spell if you informed everyone it’s just lines of code that doesn’t think. A statistically significant number of Americans claim to talk with spirits and angels, or have the gift of prophecy. On the flip side the techbro types believe in garbage like the future basilisk AI that tortures evseryone forever. These same people are already deranged, all the LLM does is organize their mashed potato brains into sentences that can be read. And just about every major LLM seems primed to have a very servile, docile writing style so it’s trivial to get them to say whatever you want so long as you keep saying the same thing. They’re not well designed for confrontation.

I firmly believe one of the best ways to deal with reactionaries, woowoo types peddling scams, or conspiracy theorists is to simply tell them they’re a fucking idiot. “That sounds fucking stupid. Shut up, nerd, and never speak to me about this again.” That’s how you do it, that’s how you dispell stuff. Social embarrassment and confrontation.

An LLM won’t do that, it’ll rub a nice mental salve on your already smooth brain. It’s an actual echo chamber.