The panic around the threat of ‘deepfakes’ began in earnest in 2018.
The ability to impersonate the voice or face of a person strikes fear into the heart of Senators, as well as the vulnerable human rights activists and journalists with whom I work at the human rights organization, WITNESS.
So far, deepfakes have been used in a few instances to attack the credibility of journalists. There are thousands of examples of non-consensual use of the faces of celebrities and public figures on pornographic websites and elsewhere.
We have not, however, seen other potential malicious usages, in terms of undermining national security, conducting broad attacks on public trust, targeting influential newsrooms, or widespread integration into influence campaigns.
Mainstream products from the major commercial players already use AI-driven approaches, like Portrait Mode on the iPhone, Night Sight on your Pixel, or filters and augmented realities in Snap. But the most dangerous functions have not yet been productized – yet.
More subtle forms of ‘synthetic media’ manipulation are emerging, like the ability to alter the background of videos, remove an object or insert a person. These synthetic media threaten the integrity of news journalism, human rights documentation, and investigative reporting.
And the weaponization of the idea that we cannot believe any image – which is simply not true in the near-term for most images we will encounter – will be a boon to authoritarians and totalitarians worldwide.
We are in the calm before the storm.
This is an opportunity to be seized. We can be proactive and pragmatic in addressing this threat to the public sphere and our information ecosystem.
We can prepare, not panic.
In this context, I’ve been leading the work at WITNESS on this emerging threat. At WITNESS, we began our exploration of this problem set by convening the first expert meeting connecting technologists, industry insiders, researchers, human rights investigators and journalists to shape the range of pragmatic, partial solutions I outline here and below.
We have followed this with leading a series of threat modelling workshops to understand how other constituencies including journalists and people working in the misinformation space perceive the risks.
The necessity for WITNESS to lead on this is clear. As a human rights network, we are focused on the power of video and technology as a tool for transparency, accountability, and rights, both in the US and internationally. We’ve been a key player in supporting many millions of people around the world to use the explosion of mobile, social media and the Internet ethically and persuasively to share their ground-truthed realities, and then advocate for change.
For many people, taking a video of an act of injustice is their first response to seeing it in front of them. Snapchat and Instagram form the lingua franca of young people. YouTube is used as the go-to place to learn a new skill for many.
We know the power of these media when used for good. It’s in the compelling evidence from the ground that courageous activists and civic journalists in Syria and Myanmar have shared of war crimes and atrocities.
It’s in the movement organizing that is seen here in the US in the coalescence of a more powerful Movement for Black Lives around the visual evidence of police violence.
Where we should start to defuse deepfakes
So, what is the solution-set we need for deepfakes and other synthetic media?
To start with, we should build on what exists already. Any response to deepfakes needs to recognize that the threats are intimately linked to other misinformation trends and mechanics, from bots and computational propaganda to the online attention economy to declining public trust in institutional information.
Similarly, we must also see how we can build on existing expertise in dealing with what we might call ‘shallowfakes’. By shallowfakes, I mean the tens of thousands of videos circulated with malicious intent worldwide right now. These videos are made not with sophisticated AI but often simply relabeled and re-uploaded, claiming an event in one place has just happened in another. Some are videos that have simple edits, or a new audio track.
Videos showing horrible atrocities crop-up repeatedly, recycled between countries. Notorious examples include the mis-contextualized, lightly edited video that incited mob violence in India, or the video claiming to show migrant violence in Europe that President Trump retweeted from British far right sources.
A strong practice has emerged in the last decade among human rights investigators and journalists of open-source intelligence (OSINT) to detect fake images and make use of compelling evidence and newsworthy footage found online. This includes robust practices to work out the veracity of images and the credibility of sources found online. Groups like Bellingcat and the Visual Investigations team at the New York Times, and companies like Storyful have been at the forefront of this work – as well as WITNESS’ own Media Lab.
Newsrooms using these OSINT approaches and the burgeoning professional and amateur OSINT community will be at forefront of organized defenses against deepfakes.
One easy next step will be to make sure that we’re coordinating and connecting these researchers, academics and technologists with journalists who already spend every day sorting the true from the false in online spaces. As those researchers build technologies we should aim for maximum compatibility and ease of use.
There is no one technical silver bullet for deepfakes. We need complementary approaches of media forensics. We need analyses of existing databases of videos to trace origins. And we need new approaches that apply the same generative adversarial network-based deep learning techniques used to create fake images to the process of detecting them.
How news organizations can collaborate
We should fear the use of these tools in upcoming elections worldwide, from India in 2019 to the United States in 2020. One thing we can do is ensure that this technical support and relationship-building is embedded in well-resourced, cross-outlet misinformation prevention efforts.
A series of these have been coordinated across the globe by groups like First Draft around the recent French, U.S., Brazilian and Mexican elections. Similar efforts should take place on a non-crisis basis for newsrooms to share information between them. Large newsrooms like the Wall Street Journal and BBC that have taken steps on this can share what they know with less well-resourced newsrooms. Otherwise, bad actors will simply target the weakest links.
A bigger, broader question is how news organizations will defend themselves and one another each other against the damage of President Donald Trump and others politicians claims that what you’re seeing and what you’re reading is not what’s happening.
One of the most serious ramifications of deepfakes and other forms of synthetic media is that they will further damage people’s trust in our shared information sphere.
This is as likely to occur because of public figures calling news they don’t like ‘deepfakes,’ claiming you cannot believe anything you see or hear, or exercising ‘plausible deniability’ on compromising images and audio, as it is from widespread actual use of deepfakes.
Tuning the tools
Next, we should ensure that we ask the right questions about the technology infrastructure we want to combat deepfakes. There is a growing field of innovation around discerning what is faked. There are also a growing range of tools that focus on proving what is real. For both areas of technological growth, real scale will come if they are integrated into consumer platforms or available as broadly used tools.
Tools that prove what is real typically focus on ensuring images have a clear provenance. They often rely on approaches such as a controlled capture environment to ensure that images and video as they are captured cannot be easily tampered with. They use strategies like the addition of rich meta data to provide a series of signals and avoid spoofing risks. Often, they include the use of a distributed media ledger to which a hash of an image can be shared at point-of-creation.
A WITNESS research project is looking at the implications of these tools. Beyond the philosophical question of how quickly we want to or need to move as societies to a technically guided ‘disbelief by default’ approach to images, we also need to consider how these systems deal with some scenarios that are particularly important for a robust civil society.
We should ask how any technology will protect people in risky situations, like dissidents, who may need to hide their identity or revoke information which later creates danger for them. For almost all these technologies, we need to ask about a disproportionate “ratchet effect,” whereby the adoption of a new technology raises the bar both technically and practically for some people who cannot afford or risk to adopt it.
From my experience with at-risk journalists and accountability activists, these people might well not be able to fully participate in this new ecosystem. And they are also the people who have spent decades being told they are ‘fake news’ before that became a buzzword. “Technologies of truth” may be used against them for delegitimization.
Rallying the platforms
Before we settle on a ‘disbelief by default’ technology approach, we need participants across the technology stack, from chipmakers to start-ups to app-developers to platforms, to engage in a meaningful dialogue with civil society and journalists about the way to do this responsibly and right.
Facebook, Google, Twitter and other regional platforms will be at the center of how much deepfake and synthetic media content is shared. Again, we need a proactive dialogue on what role we expect of them, including how we expect them to inform each other, governments, news organizations and individual users about deepfakes and other synthetic media they detect. This does not mean we should ask them to be the censors of deepfakes and synthetic media content. There will be deepfake content that run afoul of platforms’ terms of service, as well as local law such as dignitary torts.
We should hope, however, that the vast majority of synthetic media will be delightful fun, boring-to-anyone-but-the-recipient personal communication, or compelling satire. Malicious uses will hopefully be a small minority. We should avoid decisions that risk even a low level of false positives of incorrect removals. An over-broad content moderation policy would also fall foul of the recent human rights-based model for content moderation from the UN Special Rapporteur on Freedom of Expression.
This also reinforces the definitional question of what a “synthetic media item” actually is. Already, the term “deepfakes” is used to describe a far broader range of manipulations than just face impersonation.
Should “synthetic media” include background blurring like Portrait Mode or enhanced Night Sight? And what are the gradations between this and a full-on deepfake? What, when and how should a platform-based or standalone detection tool signal the presence of AI-facilitated image manipulation to users?
This will not be like other broad-based issues like child exploitation imagery, where a shared hash-based detection and automatic removal approach can generally work. There is, however, a possible technical incentive here for platforms to collaborate. For the fakes created via deep learning algorithms, the forger has the technical advantage, until there are enough examples of a particular technique to build into a detection approach.
A number of the emerging detection techniques, like the Face Forensics database embedded above, work by creating images based on the latest forgery technique and then training the AI to detect a particular model of forging. The technology companies that operate search, video-sharing and social media platforms can collaborate to share training data and to avoid bad actors exploiting the weakest link of the company with the worst detection tools.
This would overlap with the type of information that is already sometimes shared in terms of bad actor activity, like the detection of bot networks. Companies could also work out how to share meaningful signals they all detect to key stakeholders who can engage with the most damaging fakes. This need not look like the current fact-checking partnerships that have met with mixed results on platforms like Facebook.
Technology companies and others involved in consumer image production also need to be clearer about what they will or won’t release as commercial products, along with the safeguards they will include to make it easier for others to detect manipulation created with their tools.
Stepping carefully with new laws for deepfakes
There may be a need for legislation around deepfakes. There is also, however, a wealth of existing law that might be relevant to some particular usages. Bobby Chesney and Danielle Citron have explored a number of these in recent articles. Lawmakers like Senator Mark Warner have floated ideas of making another carve-out on Section 230 around malicious deepfakes that have been found in a court of law to constitute a dignitary tort.
Any legislation should be careful not to treat deepfakes in isolation from other misinformation issues. Legislators also to be aware of how the precedents of United States legislation play out other jurisdictions that may imitate it for more authoritarian, counter-free speech ends, or use the duties it imposes on companies for counter-human rights ends.
Ultimately, this is a techno-social problem as much as a technical or governance problem. In an era where it may not be possible to the human eye to detect manipulation in an image or that an image has been created with artificial intelligence, we need to think about what new literacies the public needs to grapple with more readily faked and individualized audio, video and photos.
As with any of the solutions above, literacy approaches need to build on our understanding of how people engage with and share misinformation and disinformation. We know that we are cognitively ill-equipped to discern fake images from fact, and to distinguish fabrication from remembered reality in memory. We know that an objective ‘truth’ is not the sole criterion that people use for trusting or sharing content.
A stronger research basis will help us do this, as will a better understanding of the mechanics of how to engage with people in the places they will encounter deepfakes, manipulated video and audio: in their newsfeeds, search results and in messaging groups on their phones.
Would a heat map of potentially manipulated content in an image be useful or useless? Would showing images from the same event with a different viewpoint or ‘revealing’ fakery be helpful? Would reverse video search to quickly see if a video has an existing (and different) source than claimed be useful in a closed messaging group?
As we prepare for the first wave of malicious synthetic media, face-swaps and deceptive audio, there is a concrete opportunity to be prepared in a way many people were not for the waves of deception in the last five years.
We should get started.
Sam Gregory is the program director of WITNESS, which supports anyone, anywhere to use video and technology to fight for human rights. An award-winning technologist and advocate, he leads work around emerging opportunities and threats for activism and journalism including artificial intelligence, proactive approaches to malicious ‘deepfakes,’ innovation in eyewitness video, and challenges to trust and evidence. He also supervises WITNESS’ Tech Advocacy work, which advocates to technology companies on how products protect human rights and develops tools such as ProofMode for better authentication of contentious video. He is Co-Chair of the Partnership on AI’s Working Group on Social and Societal Influences, and a member of the Technology Advisory Board of the International Criminal Court and the WEF Global Future Council on Human Rights.