On May 30, 2024, OpenAI published its first report detailing the use of its AI tools in covert influence operations. Over the past three months, the company has identified and blocked five attempts to use OpenAI’s generative AI to create and spread propaganda on social media by actors from Russia, China, Iran, and Israel.
For instance, in an operation targeting users from Ukraine, Moldova, the Baltic states, and the United States, Russian propagandists developed a Telegram bot designed to generate political comments. Actors from China created content for X and Medium, praising their government’s international efforts. In Iran, propagandists produced articles criticizing the United States and Israel. The Israeli company Stoic established fake accounts and generated various content concerning the conflict in Gaza.
Propaganda, fake news, pranks, fraud, and pornography are now actively being produced using neural networks, also known as generative artificial intelligence. AI-generated misinformation appears in every possible format: video, images, audio, and text. Deception has never been so convincing or so easy to create.
In this article, Factcheck.kz team provides guidance on combating AI-generated misinformation. We have compiled the most useful resources, including the latest tools for identifying false information, as well as tips on cyber hygiene and critical thinking. Additionally, we have summarized global legislative experiences in regulating neural networks and their generated content.
Expectation vs reality
It is well known that artificial intelligence has long been demonized in science fiction, often portrayed as a primary antagonist when it spirals out of control. However, concerns about the potential dangers of AI extend beyond literature and cinema.
Long before the emergence of Midjourney, ChatGPT, and other generative AI systems, experts predicted that the threat posed by AI would increase alongside its development. Admittedly, these warnings often sounded like predictions of an imminent machine uprising in the style of “Terminator” to many.
Hence, IT giants continued to actively invest in AI, while scientists continued advancing the field. It’s evident that generative AI has sparked a revolution poised to transform entire industries.
Yet, humanity has a knack for turning any promising technology into a weapon, and neural networks are no exception. Today, we can assert that the utilization of generative models, especially for generating disinformation, poses a very real threat.
In 2024, analysts at the World Economic Forum identified AI-generated disinformation as the most pressing short-term threat to the global community. According to experts, advanced AI programs are increasing the spread of false information, posing a significant risk to democracy, societal polarization, and representing a major immediate threat to the global economy.
In early 2023, OpenAI, in collaboration with the Stanford Internet Observatory and the Center for Security and Emerging Technologies at Georgetown University, published a report discussing the potential risks of using large language models, such as Gemini or Chat-GPT, for disinformation. The company’s analysts acknowledged that “with the growing prevalence of large language models […], it has become nearly impossible to ensure that these models will not be used to generate disinformation.”
Simultaneously, the report’s authors suggest that a potential solution to the spread of misinformation lies in the collaborative efforts of social networks, AI program developers, and governments. However, experts caution that even these combined efforts to regulate content might not be sufficient to manage the rapid pace of technological advancements.
He doesn’t need your clothes and your motorcycle
When discussing the information risks associated with generative AI, we are not referring to the programs themselves, but rather their users. Scammers, pranksters, and propagandists quickly realized that neural networks could help them achieve their objectives more swiftly and effectively.
As a result, it is now possible to create video and audio deepfakes of celebrities and politicians with ease, eliminating the need for lengthy and meticulous editing. Fraudsters are already utilizing generated videos to promote their schemes (1, 2, 3) and even deceive people out of large sums of money in real time through video calls.
Even more concerning are deepfakes used for political manipulation, as they can be employed to discredit any opponent.
This occurred with a prank targeting anti-war Russian activists, involving a deepfake video of Moldovan President Maia Sandu discussing an upcoming mobilization in the country. Another instance involved an audio deepfake of the Mayor of London calling for the postponement of an event commemorating World War II victims in favor of a pro-Palestinian march. The spread of this latter fake nearly incited mass riots.
Propagandists have also embraced generative neural networks. For instance, a certain news site linked to the Chinese government used AI to spread false news about “American biological laboratories in Kazakhstan” within the Kazakh segment of the Internet.
Since the start of the full-scale invasion of Ukraine, Russian propagandists have been using deepfakes of high-ranking Ukrainian officials. The image of Vladimir Zelensky is frequently used in these “videos,” such as in a fake announcement about Ukraine’s surrender or a call to surrender Avdiivka.
The impact of generated disinformation on the electoral process is a significant concern. In 2024, elections will take place in 50 countries, including numerous European nations and the United States. Neural networks have already been used in elections in Pakistan, Indonesia, India, and Slovakia.
So far, generated disinformation has not caused significant harm to the global electoral process. However, with the year still far from over, it is uncertain how deepfake campaigns will influence the upcoming elections.
Seeing is believing
The issue with the spread of AI-generated misinformation is not merely the increased number of fakes —though their prevalence has surged significantly in recent years — but rather the accessibility of fake-creation technology to anyone with Internet access. Additionally, these fakes are so realistic that identifying their artificial nature can be very challenging without specialized skills and tools.
Simultaneously, the virality of AI-generated fakes is so significant that, despite the swift efforts of fact-checkers, stemming the spread of misinformation remains challenging. Furthermore, even after the falsity of the message becomes evident, disinformation continues to affect worldviews and erode trust in media— a fact likely understood by the creators of such fakes.
Regrettably, cognitive biases, which lead individuals to retain false information even after it has been disproven, are not the sole aspect of our psyche that benefits AI misinformation disseminators.
In an experiment conducted by scientists at the University of Zurich, it was discovered that individuals are more inclined to believe fake messages generated by AI compared to those crafted by humans. The researchers speculate that the generated texts appear more credible due to their clearer structure and concise presentation. It’s worth noting that the margin in favor of AI was only 3%; however, researchers utilizing ChatGPT-3 in the experiment suggest that this percentage may rise with the utilization of more advanced versions of large language models.
Scientists from the Max Planck Institute and the University of Amsterdam discovered another aspect of deepfake perception, similar to the Dunning-Kruger effect. Through experiments, it was found that untrained viewers cannot distinguish real videos from deepfakes, yet they tend to rate their ability to detect fakes as high. Compounding this issue, subjects predominantly mistook fake videos for real ones, rather than vice versa.
Information literacy and prebunking are deemed effective strategies for mitigating cognitive biases and countering misinformation.
According to the Community manager at the International Fact-Checking Network (IFCN), Enock Nyariki, “In order to mitigate the harm caused by deepfakes, media literacy campaigns are indispensable. Educating the public to adopt a mindset akin to fact-checkers can notably decrease the propagation of misinformation among the general populace and diminish the motivation for those propagating it.”
AI detection programs and their accessibility
Detection programs designed to identify AI traces in videos, images, or texts serve as significant tools, yet for the average user, they may not be as practical or efficient as exercising critical thinking and attentiveness.
Currently, AI identification remain largely a professional tool for businesses, journalists, and fact checkers. Many of these programs, whether in their entirety or with expanded features, are accessible solely through corporate subscriptions or for a fee. Various factors contribute to this situation.
The creation of advanced and versatile tools for detecting generated content necessitates resources, making it understandable that companies developing AI identification technology seek to monetize their product. This is particularly evident given that investors appear to (1, 2) allocate less focus to AI footprinting technologies compared to the advancement of generative AI itself.
Another probable factor contributing to the limited availability of publicly accessible AI tools is the risk that once a tool is made public, it can be swiftly manipulated or bypassed. Consequently, developers are compelled to find a balance between the accessibility and efficacy of their product.
Furthermore, another significant factor restricting public access to specialized programs is the prerequisite for specific skills and knowledge to effectively utilize the program and interpret its outcomes.
Angie Holan, director of IFCN and former editor of PolitiFact, suggests that thorough verification of information demands a level of attention often absent among the general public. Checking disinformation sometimes necessitates specialized information retrieval skills that non-specialists typically lack. Furthermore, the average news consumer must be sufficiently motivated to utilize a paid program for detecting AI footprints.
Anyone can utilize programs like Deepware or AI or Not, but comprehending the implications of the results — especially when they differ across platforms — is a nuanced endeavor.
Certain AI detection tools present the outcome as a percentage probability, whereas others categorize the result into various classifications, such as “most likely created by AI” or “most likely created by a human”. Additionally, some programs employ more sophisticated visualizations like graphs and heat maps, which may pose challenges in interpretation without supplemental training.
When it comes to the efficacy of these tools, there’s often room for improvement. Technical tools, in particular, face challenges in accurately identifying generated text and audio. For instance, AI detectors may yield disparate results when paragraphs in the text are swapped or when an audio file has undergone multiple downloads and uploads. Hence, for a dependable identification of deepfakes, one should not solely rely on specialized AI identification programs but also utilize other methods, including attentiveness, critical thinking, and familiarity with AI markers across different modalities.
If you find yourself needing assistance from technical analyzers, we can suggest the following detectors based on our practical experience.
We selected these programs due to their accessibility, user-friendliness, and relative effectiveness. However, it’s important to note that none of these analyzers are perfect, and it’s advisable not to solely rely on their assessments. Instead, it’s recommended to analyze images, audio, or video using additional means and methods.
- AI or Not is a tool that examines both images and audio. In our observation, it performs well with images, but its proficiency with audio deepfakes is less assured. Registration allows for a free basic verification of 10 images and audio files, with subscription plans starting at $5 per month thereafter.
- Undetectable AI is a tool designed to examine text for indications of AI involvement, revealing the programs likely used to generate the text. It’s accessible without registration. However, it has a tendency to produce false positive results.
- DeepFake-o-Meter is a versatile tool capable of analyzing images, audio, and video. It offers the flexibility to choose from a range of detectors developed by various providers. Users have the option to avoid storing scanned files on the server, which is crucial for handling sensitive information. The tool is free and accessible after a simple registration process. However, not all detectors are equally effective at identifying generated content. Nonetheless, in certain instances, the program has outperformed its competitors.
- Deepware is a tool designed to detect deepfakes in videos. Users have the option to upload their own files or input links from platforms like YouTube, Facebook, or X. Serving as an aggregator, it presents results from various detectors as a percentage representing the likelihood of AI involvement. While it may not consistently provide a reliable average result, it demonstrates effectiveness in identifying viral fakes.
- The AI Speech Classifier from ElevenLabs is proficient in identifying audio generated by the ElevenLabs neural network. It can be accessed without registration and offers unlimited checks. While our experience indicates that the analyzer generally performs well, its effectiveness may not always be consistent. This was evidenced when the program failed to detect AI traces in a fake voice recording of the US President, suggesting occasional limitations in its detection capabilities.
Eagle eye
While advancements in generative AI pose a threat of making it increasingly challenging to differentiate between generated and authentic content, current distributors of fakes often do not prioritize creating high-quality fakes.
Many viral deepfake videos and generated images can be easily identified with the naked eye. Examples of such fakes include “photos” purportedly showing French President Emmanuel Macron during the riots in Paris, a recent fake featuring Donald Trump surrounded by his black supporters, and a fake video “interview” of the Secretary of the National Security and Defense Council of Ukraine, Oleksiy Danilov, allegedly confirming Ukraine’s involvement in a terrorist attack at Moscow’s Crocus City Hall.
The spread of these messages is not primarily due to the high quality of the fakes, but rather the viewers’ inattention, their inclination to seek confirmation of their own viewpoints, and their tendency to share sensational news without verifying its accuracy. The creators of these fakes are cognizant of these vulnerabilities in our perception and capitalize on the swift dissemination of fakes, with the mediocre quality of the fakes not hindering this process.
Here, information literacy proves valuable once again. While this article won’t delve into the intricacies of detecting AI fakes, plenty of resources exist on this topic (for example: how to identify generated audio, images, or video deepfakes). Nonetheless, combating AI-generated misinformation can be achieved by adhering to a couple of straightforward guidelines:
- Thoroughly scrutinize the material you intend to share, paying close attention to image details. Obvious indicators of AI manipulation, such as an additional hand in an image or unnatural facial expressions in a video, may go unnoticed if news is merely skimmed over.
- Verify the credibility of the message sources.
- Don’t hesitate and employ the lateral reading method to cross-reference the information — it’s possible that fact-checkers have already debunked it.
Artificial Intelligence Control Measures
Disinformation created using neural networks presents a relatively new challenge for the global community. Consequently, the legal framework for the use of generative AI, both domestically and internationally, is still being developed, and its effectiveness will become apparent only over time.
Unfortunately, existing legislation does not currently prevent the spread of fakes, but instead acts reactively. Analysts in economics and IT suggest (1, 2) that any measures to control AI will always lag behind technological advancements.
However, as experts from Google emphasize, “AI is too important [a technology] not to regulate — and too important not to regulate well.”
IT specialist and director of the Internet Defense Society, Mikhail Klimaryov, believes that it is technically impossible to limit the spread of disinformation generated by neural networks or to hold the creators accountable, as they are often based in different countries from where the disinformation is spread. However, companies can be legally mandated to monitor the quality of information disseminated on their platforms or through technologies they control.
Major social media platforms such as X, YouTube, Facebook, and TikTok have implemented various measures to control generated or modified content. In February, Google, OpenAI, TikTok, Meta, and Microsoft proposed a draft agreement to jointly develop strategies to combat “election disinformation created by artificial intelligence.” By early spring, Meta and YouTube announced expanded rules for labeling generated content, and Google blocked users from asking election-related questions in its Gemini chat.
National Laws and Policies
One of the earliest legislative acts indirectly related to artificial intelligence was introduced in South Korea in 2008, long before the prominent emergence of generative models. This document aimed at regulating the development and spread of “smart robots.” Over time, other countries have gradually established guidelines and regulations to control AI technologies across various fields (1, 2).
The first country to publish a National Strategy for the Development of Artificial Intelligence was Canada. The document outlines four goals and corresponding implementation programs, with the top priority being the support of research and talent.
Almost simultaneously with Canada, China adopted its “Next Generation Artificial Intelligence Development Plan.” This is the most extensive and comprehensive AI project to date.
The authors of the Chinese plan emphasize that the development of artificial intelligence could pose security threats, which need to be addressed. In 2018, China released the White Paper on AI Standardization (the full text of the 2021 edition is available here). This document, in addition to implementation and management initiatives, also covers reliability and security issues. Consequently, in the field of AI, there are both projects focused on scaling and development and those focused on assessment and regulation.
In 2020, the international agency HolonIQ reported the existence of 50 national strategies for artificial intelligence. That same year, the OECD AI Policy Observatory was officially launched. To date, OECD AI has documented over 1,000 AI policy initiatives in 69 countries worldwide. Of these, 354 are focused on AI guidance and regulation, though it is important to note that most are generally advisory in nature.
European Union: AI Act as a Model Law
The European Union has arguably made the most significant strides in legally regulating non-recommendation type AI.
In 2019, the European Commission released the dossier “Strengthening Trust in Human-Centered Artificial Intelligence,” which outlines guidelines for creating trustworthy AI. Key requirements for AI applications include:
- Human activity and control
- Technical reliability and safety
- Privacy and data governance
- Transparency
- Diversity, non-discrimination, and fairness
- Social and environmental well-being
- Accountability
In December 2023, EU countries and the European Parliament reached an agreement on artificial intelligence legislation. In March 2024, the European Parliament adopted the law, known as the AI Act. This legislation is already being hailed as a landmark and is expected to become the foundation for a new governance model built on technology.
The project aims to classify and regulate AI applications by dividing them into four categories based on their level of risk. According to the regulations, some systems will be completely prohibited (such as certain real-time surveillance technologies). Others will undergo rigorous assessment, while some will not be subject to supervision at all.
The document also stipulates that artificial or manipulated images, audio, or video content (“deepfakes”) must be distinctly labeled as such.
USA
Another significant development in the legal realm is the Decree on the development and use of artificial intelligence, signed in 2023 in the United States. The White House website describes the initiative as establishing “new standards for AI safety and security, safeguarding Americans’ privacy, promoting fairness and civil rights, protecting consumers and workers, fostering innovation and competition, bolstering American leadership globally, and more.”
Thus, according to the new regulations, developers of AI systems that pose a threat to US national security, the economy, public health, or safety are required to provide the government with safety test results. The Department of Homeland Security has been assigned the responsibility of establishing an AI Security Council and collaborating with the Department of Energy to address threats pertaining to artificial intelligence, cybersecurity, as well as chemical, biological, radiological, and nuclear risks. Additionally, to safeguard Americans from AI-based fraud, the Commerce Department will devise guidelines on content authentication and watermarking to accurately identify AI-generated content.
Moreover, in early February 2024, the US government introduced a ban on robocalls utilizing AI-generated voices. The restriction was enacted following an incident in New Hampshire, where residents started receiving an influx of calls allegedly from President Joe Biden. These calls featured a generated voice of the president urging his supporters not to vote in the presidential primary.
What about Kazakhstan?
As of now, Kazakhstan does not have specific laws aimed at regulating the creation and operation of AI. All national initiatives identified by OECD AI are solely focused on fostering the development of projects related to artificial intelligence.
The necessity for legal regulation of AI and robotics is outlined in the Concept of Legal Policy of the Republic of Kazakhstan until 2030. Lawmakers highlight the primary concerns as being the allocation of responsibility for harm resulting from the actions of AI and robots, along with determining ownership of intellectual property rights to works created with their involvement.
Conclusion
There is no universal solution for combating disinformation generated by neural networks. Effective resistance requires joint efforts involving social networks, AI developers, journalists, governments, and the enhancement of media literacy among the populace.
The latter is akin to immunization through vaccination and the development of collective immunity. Only through awareness-raising and fostering critical thinking skills can the proliferation and influence of AI-generated fakes be reduced, thus safeguarding the information ecosystem and democracy.
This article was published with the support of “The Exchange”
You must be logged in to post a comment.