Sam Altman’s remarks came during a rare discussion with reporters over dinner.He spoke about ChatGPT’s rapid growth, the possibility of encrypting chats for better privacy and the new browser wars.
and the seed lottery. You can see this if you try training a simple network with two inputs to learn xor. It can converge in multiple ways, and sometimes it converges to a really bad approximation. And sometimes it doesn’t converge at all (or it converges so slowly that it might as well be considered not to converge). And even then it might still converge to an approximation that’s more accurate on one side of the input space than the other. Tons of ways to get an undesirable result. For a simple 2-input network.
Imagine how unlikely it is for txese models to actually converge to the optimal thing. And how often the training is for nothing.
and the seed lottery. You can see this if you try training a simple network with two inputs to learn xor. It can converge in multiple ways, and sometimes it converges to a really bad approximation. And sometimes it doesn’t converge at all (or it converges so slowly that it might as well be considered not to converge). And even then it might still converge to an approximation that’s more accurate on one side of the input space than the other. Tons of ways to get an undesirable result. For a simple 2-input network.
Imagine how unlikely it is for txese models to actually converge to the optimal thing. And how often the training is for nothing.
I am sure they weed out bad seeds but in the end, it is still kind of random
yes, but the only way to weed out a bad seed is to “play the lottery”
By the time you discover a seed is bad, you’ve already spent a shitton on training. Money down the drain, you gotta start over
Just train another AI to train AI, then train another and another, and another. Imagine the stock rally