Post #73672 by wzambon on the US Policy board

Halls of Shrewd'm / US Policy

Unthreaded | Threaded | Whole Thread (2)

Post New | Post Reply | Report Post | Recommend It!

No. of Recommendations: 4

The science of why AI makes shit up

Researchers at Georgia Tech and OpenAI (ironically) studiedą why AI models often hallucinate.

The best way to think about this: Think of AI like a college student taking a test.

They do their best on every question. Some questions they know for sure. But when a question arises that they don’t know, they don’t write “I don’t know” in the answer box. They give an answer—because that gives them at least a chance of being right and scoring points.

The scientists found:

AI models make things up because they are trained to produce the most likely next words, not to verify the truth. “Like students facing hard exam questions, large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty,” the researchers wrote.
Some hallucinations are unavoidable because many facts are sparse, arbitrary, or don’t follow a clear pattern in the training data.
How the models are trained and graded is lacking. Most tests of the models reward correct guesses but give no credit for admitting uncertainty—e.g. they get no points for “I don’t know.” So models are pushed to answer confidently rather than admitting uncertainty.

The takeaway: hallucinations are a built-in risk of current language models, and reducing them will require better incentives for honesty.

https://www.twopct.com/p/the-expedition-march-26-e...

Post New | Post Reply | Report Post | Recommend It!

Print the post

Unthreaded | Threaded | Whole Thread (2)

Prev | Next

Announcements

US Policy FAQ

Contact Shrewd'm
Contact the developer of these message boards.

A community forum supporting civilized, and highly helpful, self-educating investors that come together for Shrewdness and merry spirits. These message boards closely follow the look and feel of the old message boards at boards.fool.com prior to the boards redirected to discussion.fool.com. Shrewd'm is not affiliated with the comprehensive and excellent investment website, The Motley Fool (TMF), in any way. The Shrewd commmunity here has owed, and continues to owe, gratitude to The Motley Fool for nurturing a culture of jovial irreverence towards Wall St, and that tradition will continue here.