DISCOVER: A Data-Driven Integrated Approach for Semantic Inconsistencies Verification

Semantic Forensics (SemaFor)


Media manipulation tools and generation techniques have rapidly proliferated the digital world. For example, the advent of GANs has led to more realistic and seamless manipulations that can fool even experts. However, these approaches rely on large amounts of data and often miss semantic details when attempting complex manipulations and generation. The US Department of Defense (DoD) would like to be able to extract knowledge from media assets and use semantic information to detect manipulations. The goal of this research project is to develop a set of forensics tools to determine the integrity, semantic consistency, source, and intent of media assets such as images, videos, and documents.

We have assembled a team of outstanding technical experts from six universities, with complementary skills and backgrounds in computer vision and biometrics, machine learning, digital forensics, as well as signal processing and information theory. We are investigating all areas of media integrity.

This work is supported by the Defense Advanced Research Projects Agency (DARPA) as part of the Semantic Forensics (SemaFor) Program.

Principal Investigators

Mauro Barni (Co-Principal Investigator), Full Professor and President of the degree course in Electronics and Telecommunication Engineering, University of Siena

Paolo Bestagini (Co-Principal Investigator), Assistant Professor at the Department of Electronics, Information and Bioengineering, Politecnico di Milano

Kevin W. Bowyer (Co-Principal Investigator), Schubmehl-Prein Professor in the Department of Computer Science and Engineering, University of Notre Dame

Edward J. Delp (Principal Investigator), The Charles William Harrison Distinguished Professor of Electrical and Computer Engineering and Professor of Biomedical Engineering, Purdue University

Giovanni Poggi (Co-Principal Investigator), Full Professor at the Department of Electrical Engineering and Information Technologies, University Federico II of Naples

Anderson Rocha (Co-Principal Investigator), Professor at the Institute of Computing, University of Campinas, Brazil

Walter J. Scheirer (Co-Principal Investigator), Associate Professor in the Department of Computer Science and Engineering, University of Notre Dame

Benedetta Tondi (Co-Principal Investigator), Research Associate at the Department of Information Engineering and Mathematics, University of Siena

Stefano Tubaro (Co-Principal Investigator), Full Professor and Coordinator of the Telecommunications Section of the Department of Electronics, Information and Bioengineering, Politecnico di Milano

Luisa Verdoliva (Co-Principal Investigator), Associate Professor at the Department of Industrial Engineering, University Federico II of Naples

Tim Weninger (Co-Principal Investigator), Frank M. Freimann Collegiate Associate Professor of Engineering in the Department of Computer Science and Engineering in the College of Engineering, University of Notre Dame

This material is based on research sponsored by the Defense Advanced Research Projects Agency (DARPA) and the Air Force Research Laboratory (AFRL) under agreement number FA8750-20-2-1004. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DARPA and AFRL or the U.S. Government.