Meta uncovers AI-generated disinformation networks on Facebook and Instagram

Meta revealed that it had detected “likely AI-generated” content on Facebook and Instagram. These posts appeared under global news articles and messages from U.S. lawmakers. 

This discovery is significant because, while Meta has identified AI-generated profile photos in influence operations since 2019, this is the first report to detail the use of advanced generative AI technologies since their emergence in late 2022. 

Generative AI, capable of producing human-like text, imagery, and audio quickly and cheaply, raises concerns about more effective disinformation campaigns potentially influencing elections.

Meta security executives noted that these advanced AI tools haven’t hindered their ability to disrupt such networks, as the AI-generated content hasn’t been realistic enough to be confused with authentic photos of politicians. Mike Dvilyanski, Meta’s head of threat investigations, mentioned that generative AI allows for faster and more voluminous content creation but hasn’t affected detection capabilities.

In the first quarter, Meta disrupted six covert influence operations, including the STOIC network. Another disrupted network was based in Iran and focused on the Israel-Hamas conflict, though it did not use generative AI. 

Meta and other tech companies are grappling with the misuse of new AI technologies, especially regarding elections. Despite having policies against disinformation, AI image generators from companies like OpenAI and Microsoft have produced voting-related falsehoods. These companies promote digital labeling systems for AI-generated content, but their effectiveness, particularly for text, remains questionable.