DeepMind Unveils SAFE: Revolutionising fact-checking for AI-generated content

DeepMind, a subsidiary of Google specialising in artificial intelligence, has introduced SAFE, an innovative AI-powered tool designed to fact-check responses generated by Large Language Models (LLMs) like ChatGPT. LLMs have garnered attention for their ability to generate content, but their accuracy has been a persistent concern, necessitating manual verification of their outputs.

The SAFE system developed by DeepMind addresses this challenge by automatically scrutinising LLM-generated responses for inaccuracies. It operates by decomposing the claims or facts presented in the response and leveraging Google Search to identify potential verification sources. 

By comparing the LLM’s response with information gathered from these sources, SAFE determines the accuracy of the generated content.

In extensive testing, DeepMind’s researchers evaluated SAFE’s performance by verifying approximately 16,000 facts derived from various LLM responses. 

Comparisons against human fact-checkers revealed that SAFE aligned with human assessments 72% of the time. Remarkably, in instances of disagreement between SAFE and human checkers, SAFE proved correct 76% of the time.

The availability of SAFE’s code on GitHub offers accessibility to its capabilities, allowing interested parties to utilise this tool for fact-checking purposes. 

By leveraging advanced AI techniques and the vast resources of the internet, SAFE represents a significant step forward in enhancing the reliability of content generated by LLMs.

DeepMind’s development of SAFE not only addresses the pressing issue of accuracy in LLM-generated content but also underscores the potential for AI-driven solutions to complement human efforts in information verification. 

With its transparency and accessibility through open-source distribution, SAFE stands as a valuable resource in the ongoing pursuit of trustworthy AI-generated content.