Oxford researchers develop a model to detect AI hallucinations

A recent study by researchers at the University of Oxford introduces a new statistical model to identify when generative AI is likely to ‘hallucinate'”‘ – a term used to describe instances where AI invents facts because it does not know the answer. The growing concern of AI hallucinations, especially given the sophisticated conversational abilities of large language models (LLMs) which can convincingly present false information as fact.

Hallucinations in generative AI are particularly problematic as more students and professionals rely on these tools for research and assignments. The risk is heightened in critical fields like medicine and law, where incorrect information can have serious consequences. Industry experts and AI scientists are therefore advocating for measures to mitigate these risks.

The researchers’ model can differentiate between scenarios where an AI model is certain about an answer and when it is fabricating information. Dr Sebastian Farquhar, a study author, explained that LLMs can articulate the same content in numerous ways, complicating the detection of their certainty. The new method, however, distinguishes between a model’s uncertainty about the content versus its uncertainty about the phrasing.

Dr Farquhar noted that while this advancement addresses specific reliability issues, it is not a comprehensive solution. The method does not catch systematic errors where the model consistently provides incorrect information with confidence. The study marks a significant step toward improving AI reliability, but acknowledges the ongoing need for further research to address systematic AI errors and enhance overall trustworthiness.