The original article has already been posted here, the discussion there might be interesting if someone can find it. The Register's click/ragebait framing isn't very interesting.
I don't really think people want these things to say "I don't know" - they want it to know.
That's obviously not reasonable for everything but I bet a lot of hallucinations ARE things where the model should be able to know or figure out. Most people are asking questions with well-known answers.
But I would guess the OpenAI post is correct, fundamentally they are trained in a way that rewards guessing, which I think must make it more likely it guesses even if the answer is in its reach.
This is why I just can't fully get behind the idea that AI is going to be the panacea the hype merchants keep claiming, at least in this era: making stuff up incentivizes users to prompt again, while returning "I don't know" tends to be a conversation ender. They're not incentivized to be accurate, they're incentivized to give the appearance of competence to the point that it keeps users coming back.
Tufte said it best: There are only two industries that call their customers 'users': illegal drugs and software.
The original article has already been posted here, the discussion there might be interesting if someone can find it. The Register's click/ragebait framing isn't very interesting.
I don't really think people want these things to say "I don't know" - they want it to know.
That's obviously not reasonable for everything but I bet a lot of hallucinations ARE things where the model should be able to know or figure out. Most people are asking questions with well-known answers.
But I would guess the OpenAI post is correct, fundamentally they are trained in a way that rewards guessing, which I think must make it more likely it guesses even if the answer is in its reach.
This is why I just can't fully get behind the idea that AI is going to be the panacea the hype merchants keep claiming, at least in this era: making stuff up incentivizes users to prompt again, while returning "I don't know" tends to be a conversation ender. They're not incentivized to be accurate, they're incentivized to give the appearance of competence to the point that it keeps users coming back.
Tufte said it best: There are only two industries that call their customers 'users': illegal drugs and software.