Human vs. Automatic Detection of Deepfake Videos Over Noisy Channels

Swaroop Shankar Prasad, Ofer Hadar, Thang Vu, Ilia Polian

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Identification of DeepFake video content is a challenging scientific problem that addresses a growing societal concern. We investigate the relationship between DeepFake detection by humans and by automatic methods based on state-of-the-art deep learning algorithms. The main novelty of our work is the consideration of videos that are transmitted through noisy channels and arrive with distortions. This reflects many practical environments, including surveillance based on cameras connected via noisy wireless links and videoconferencing in driving vehicles. We conduct a user study with 192 probands who classify real (genuine) and DeepFake videos with and without various classes of distortions. We find that today's deep neural networks (DNNs) outperform humans by far, whereas humans are heavily distracted by random noise from the channel. Moreover, DNNs are robust under distortions, achieving perfect classification on distorted data even when trained on distortion-free content. It appears that the human visual system and DNNs are approaching the DeepFake classification problem quite differently and their respective strengths and weaknesses are largely uncorrelated.

Original languageEnglish
Title of host publicationICME 2022 - IEEE International Conference on Multimedia and Expo 2022, Proceedings
PublisherIEEE Computer Society
ISBN (Electronic)9781665485630
DOIs
StatePublished - 1 Jan 2022
Event2022 IEEE International Conference on Multimedia and Expo, ICME 2022 - Taipei, Taiwan, Province of China
Duration: 18 Jul 202222 Jul 2022

Publication series

NameProceedings - IEEE International Conference on Multimedia and Expo
Volume2022-July
ISSN (Print)1945-7871
ISSN (Electronic)1945-788X

Conference

Conference2022 IEEE International Conference on Multimedia and Expo, ICME 2022
Country/TerritoryTaiwan, Province of China
CityTaipei
Period18/07/2222/07/22

Keywords

  • Deep Learning
  • DeepFake Detection
  • Noisy Channels

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications

Fingerprint

Dive into the research topics of 'Human vs. Automatic Detection of Deepfake Videos Over Noisy Channels'. Together they form a unique fingerprint.

Cite this