DISCOVER: A Data-Driven Integrated Approach for Semantic Inconsistencies Verification

Semantic Forensics (SemaFor)

Publications

Journal Papers

  1. M. Barni, Q. -T. Phan, and B. Tondi, "Copy Move Source-Target Disambiguation Through Multi-Branch CNNs", IEEE Transactions on Information Forensics and Security, vol. 16, pp. 1825-1840, 2021. doi: 10.1109/TIFS.2020.3045903
  2. A. Ferreira, N. Purnekar, and M. Barni, "Ensembling Shallow Siamese Neural Network Architectures for Printed Documents Verification in Data-Scarcity Scenarios", IEEE Access, vol. 9, pp. 133924-133939, 2021. doi: 10.1109/ACCESS.2021.3110297
  3. Y. Niu, B. Tondi, Y. Zhao, R. Ni, and M. Barni, "Image Splicing Detection, Localization and Attribution via JPEG Primary Quantization Matrix Estimation and Clustering", IEEE Transactions on Information Forensics and Security, vol. 16, pp. 5397-5412, 2021. doi: 10.1109/TIFS.2021.3129654
  4. J. Brogan, A. Bharati, D. Moreira, A. Rocha, K. Bowyer, P. Flynn, and W. Scheirer, "Fast Local Spatial Verification for Feature-Agnostic Large-Scale Image Retrieval", IEEE Transactions on Image Processing, vol. 30, pp. 6892-6905, 2021. doi: 10.1109/TIP.2021.3097175
  5. A. Bharati, D. Moreira, P. Flynn, A. Rocha, K. Bowyer, and W. Scheirer, "Transformation-Aware Embeddings for Image Provenance", IEEE Transactions on Information Forensics and Security, vol. 16, pp. 2493-2507, 2021. doi: 10.1109/TIFS.2021.3050061
  6. M. Yankoski, W. Scheirer, and T. Weninger, "Meme warfare: AI countermeasures to disinformation should focus on popular, not perfect, fakes", Bulletin of the Atomic Scientists, vol. 77:3, pp. 119-123, 2021. doi: 10.1080/00963402.2021.1912093
  7. D. Moreira, J. P. Cardenuto, R. Shao, S. Baireddy, D. Cozzolino, D. Gragnaniello, W. Abd-Almageed, P. Bestagini, S. Tubaro, A. Rocha, W. Scheirer, L. Verdoliva, and E. J. Delp, "SILA: A System for Scientific Image Analysis", Scientific Reports, vol. 12, pp. 18306, October 2022. doi: 10.1038/s41598-022-21535-3
  8. L. Abady, J. Horváth, B. Tondi, E. J. Delp, and M. Barni, "Manipulation and generation of synthetic satellite images using deep learning models", Journal of Applied Remote Sensing, vol. 16:4, November, 2022. doi: 10.1117/1.JRS.16.046504
  9. E. D. Cannas, N. Bonettini, S. Mandelli, P. Bestagini, and S. Tubaro, "Amplitude SAR Imagery Splicing Localization", IEEE Access (submitted), 2022.
  10. J. Wang, B. Tondi, and M. Barni, "An Eyes-Based Siamese Neural Network for the Detection of GAN-Generated Face Images", Frontiers in Signal Processing, vol. 2, pp. 918725, July 2022. doi: 10.3389/frsip.2022.918725
  11. J.Wang, O. Alamayreh, B. Tondi, A. Costanzo, and M. Barni," Detecting Deepfake Videos in Data Scarcity Conditions by Means of Video Coding Features", APSIPA Transactions on Signal and Information Processing, vol. 11:2, December 2022. doi: 10.1561/116.00000032
  12. L. Abady, E. D. Cannas, P. Bestagini, B. Tondi, S. Tubaro, and M. Barni, " An Overview on the Generation and Detection of Synthetic and Manipulated Satellite Images", APSIPA Transactions on Signal and Information Processing, vol. 11:1, November 2022. doi: 10.1561/116.00000142
  13. L. Abady, M. Barni, A. Garzelli, and B. Tondi, "GAN Generation of Synthetic Multispectral Satellite Images", Image and Signal Processing for Remote Sensing, vol. 11533, pp. 115330, September 2020. doi: 10.1117/12.2575765

Conference Papers

  1. D. Cozzolino, A. Roessler, J. Thies, M. Niessner, and L. Verdoliva, "ID-Reveal: Identity-aware Deepfake Video Detection", Proceedings of the IEEE International Conference on Computer Vision, October 2021. arXiv: 2012.02512v2
  2. D. Cozzolino, J. Thies, A. Rössler, M. Nießner, and L. Verdoliva, "SpoC: Spoofing Camera Fingerprints", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 990-1000, June 2021. doi: 10.1109/CVPRW53098.2021.00110 arXiv:1911.12069
  3. Z. Xiang, J. Horváth, S. Baireddy, P. Bestagini, S. Tubaro, and E. J. Delp, "Forensic Analysis of Video Files Using Metadata", Proceedings of the IEEE Computer Vision and Pattern Recognition Workshop, Virtual, June 2021. doi: CVPRW53098.2021.00115 arXiv: 2105.06361
  4. W. Li, B. Tondi, R. Ni, and M. Barni, "Increased-confidence Adversarial Examples for Deep Learning Counter-Forensics", Proceedings of the International Conference on Pattern Recognition, pp. 411-424, Springer, Cham, 2021. doi: 10.1007/978-3-030-68780-9_34 arXiv: 2005.06023
  5. O. Alamayreh and M. Barni, "Detection of GAN-Synthesized street videos", Proceedings of the European Signal Processing Conference, pp. 811-815, Dublin, Ireland, August 2021. doi: 10.23919/EUSIPCO54536.2021.9616262
  6. D. Gragnaniello, D. Cozzolino, F. Marra, G. Poggi, and L. Verdoliva, "Are GAN Generated Images Easy to Detect? A Critical Analysis of the State-Of-The-Art", Proceedings of the IEEE International Conference on Multimedia and Expo, pp. 1-6, July 2021. doi: 10.1109/ICME51207.2021.9428429 arXiv: 2104.02617
  7. D. Cozzolino, D. Gragnaniello, G. Poggi, and L. Verdoliva, "Towards Universal GAN Image Detection", Proceedings of the IEEE International Conference on Visual Communications and Image Processing, December 2021. arXiv: 2112.12606
  8. B. Hosler, D. Salvi, A. Murray, F. Antonacci, P. Bestagini, S. Tubaro, and M. C. Stamm, "Do Deepfakes Feel Emotions? A Semantic Approach to Detecting Deepfakes Via Emotional Inconsistencies", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 1013-1022, 2021. doi: 10.1109/CVPRW53098.2021.00112
  9. N. Bonettini, E. D. Cannas, S. Mandelli, L. Bondi, P. Bestagini, and S. Tubaro, "Video Face Manipulation Detection Through Ensemble of CNNs", Proceedings of the International Conference on Pattern Recognition, 2021. doi: 10.1109/ICPR48806.2021.9412711 arXiv: 2004.07676
  10. L. Bondi, E. Daniele Cannas, P. Bestagini, and S. Tubaro, "Training Strategies and Data Augmentations in CNN-based DeepFake Video Detection", Proceedings of the IEEE International Workshop on Information Forensics and Security, pp. 1-6, 2020. doi: 10.1109/WIFS49906.2020.936090
  11. S. Mandelli, N. Bonettini, P. Bestagini, and S. Tubaro, "Training CNNs in Presence of JPEG Compression: Multimedia Forensics vs Computer Vision", Proceedings of the IEEE International Workshop on Information Forensics and Security, pp. 1-6, 2020. doi: 10.1109/WIFS49906.2020.9360903
  12. E. R. Bartusiak and E. J. Delp, “Frequency Domain-Based Detection of Generated Audio”, Proceedings of the Media Watermarking, Security, and Forensics Conference, IS&T Electronic Imaging Symposium, pp. 273-1-273-7, Burlingame, CA (Virtual), January 2021. doi: 10.2352/ISSN.2470-1173.2021.4.MWSF-273
  13. J. Horvath, D. Mas Montserrat, and E. J. Delp, "Nested Attention U-Net(NAU-N): A Splicing Detection Method for Satellite Images", Pattern Recognition, ICPR International Workshops and Challenges, pp. 516-529, February 2021. doi: 10.1007/978-3-030-68780-9_41
  14. J. Horváth, S. Baireddy, H. Hao, D. M. Montserrat, and E. J. Delp, "Manipulation Detection in Satellite Images Using Vision Transformer", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshop on Media Forensics, pp. 1032-1041, July 2021. doi: 10.1109/CVPRW53098.2021.00114 arXiv: 2105.06373
  15. E. R. Bartusiak and E. J. Delp, "Synthesized Speech Detection Using Convolutional Transformer-Based Spectrogram Analysis", Proceedings of the IEEE Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, California, October 2021. doi: 10.1109/IEEECONF53345.2021.9723142 arXiv: 2205.01800
  16. J. Horváth, D. M. Montserrat, H. Hao, and E. J. Delp, "Manipulation Detection in Satellite Images Using Deep Belief Networks", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Workshop on Media Forensics ,Seattle, Washington, pp. 2832-2840, June 2020. doi: 10.1109/CVPRW50498.2020.00340
  17. D. Mas Montserrat, H. Hao, S. K. Yarlagadda, S. Baireddy, R. Shao, J. Horváth, E. Bartusiak, J. Yang, D. Guera, F. Zhu, and E. J. Delp, "Deepfakes Detection with Automatic Face Weighting", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Workshop on Media Forensics, pp. 2851-2859, Seattle, WA, June 2020. doi: 10.1109/CVPRW50498.2020.00342
  18. D. M. Montserrat, J. Horváth, S. K. Yarlagadda, F. Zhu, and E. J. Delp, "Generative Autoregressive Ensembles for Satellite Imagery Manipulation Detection", Proceedings of the IEEE International Workshop on Information Forensics and Security, pp. 1-6, 2020. doi: 10.1109/WIFS49906.2020.9360909
  19. W. Theisen, J. Brogan, P. B. Thomas, D. Moreira, P. Phoa, T. Weninger, and W. Scheirer, "Automatic Discovery of Political Meme Genres with Diverse Appearances", Proceedings of the AAAI International Conference on Web and Social Media, June 2021. arXiv: 2001.06122
  20. E.D. Cannas, S. Baireddy, E. R. Bartusiak, S.K. Yarlagadda, D. Mas Montserrat, P. Bestagini, S. Tubaro, and E.J. Delp, “Open-Set Source Attribution for Panchromatic Satellite Imagery”, Proceedings of the IEEE International Conference on Image Processing, pp. 3038-3042, Anchorage, AK, USA, September 2021. doi: 10.1109/ICIP42928.2021.9506600
  21. T. W. Ford, “It’s time to address facial recognition, the most troubling law enforcement AI tool”, Bulletin of the Atomic Scientists, November 2021. link
  22. T. W. Ford, M. Yankoski, W. Theisen, T. Henry, F. Khashman, K. R. Dearstyne, and T. Weninger, “MEWS: real-time social media manipulation detection and analysis”, Demonstration in Conference on Neural Information Processing Systems, Virtual, December 2021. link
  23. Z. Xiang, P. Bestagini, S. Tubaro, and E. J. Delp, "Forensic Analysis and Localization of Multiply Compressed MP3 Audio Using Transformers", Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 2929-2933, 2022. doi: 10.1109/ICASSP43922.2022.9747639
  24. K. Bhagtani, A. K. S. Yadav, E. R. Bartusiak, Z. Xiang, R. Shao, S. Baireddy, and E. J. Delp, "An Overview of Recent Work in Multimedia Forensics", Proceedings of the IEEE International Conference on Multimedia Information Processing and Retrieval, August 2022. doi: 10.1109/MIPR54900.2022.00064 arxiv:2204.12067
  25. J. Wang, O. Alamayreh, B. Tondi, and M. Barni, "An Architecture for the detection of GAN-generated Flood Images with Localization Capabilities", Proceedings of the IEEE Image Video and Multidimensional Signal Processing Workshop, pp. 1-5, Nafplio, Greece, June 2022. doi: 10.1109/IVMSP54334.2022.9816324 arxiv:2205.07073
  26. A. Pianese, D. Cozzolino, G. Poggi, and L. Verdoliva, "Deepfake audio detection by speaker verification", Proceedings of the IEEE Workshop on Information Forensics and Security, December 2022. arxiv:2209.14098
  27. H. Maaren, D. Vanden Bussche, F. Guillaro, D. Cozzolino, G. Van Wallendael, P. Lambert, and L. Verdoliva, "Comprint: Image Forgery Detection and Localization using Compression Fingerprints", Proceedings of the MultiMedia FORensics in the WILD Workshop in conjunction with IEEE ICPR, Montreal, Canada, August 2022. arxiv:2210.02227
  28. R. Corvi, D. Cozzolino, G. Zingarini, G. Poggi, K. Nagano, and L. Verdoliva, "On the detection of synthetic images generated by diffusion models", Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Rhodes Island, Greece, June 2023. arxiv:2211.00680
  29. W. Theisen, D. Cedre, Z. Carmichael, D. Moreira, T. Weninger, and W. Scheirer, "Motif Mining: Finding and Summarizing Remixed Image Content", Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, January 2023. arxiv:2203.08327
  30. L. Abady, G. M. Dimitri, and M.Barni, "Detection and Localization of GAN Manipulated Multi-spectral Satellite Images", Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, October 2022. doi: 10.14428/esann/2022.ES2022-39
  31. E. R. Bartusiak and E. J. Delp, "Synthesized Speech Detection Using High-Resolution Spectrogram Neural Analysis", Proceedings of the IEEE Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, California, October 2021.
  32. F. Guillaro, D. Cozzolino, A. Sud, N. Dufour, and L. Verdoliva, "TruFor: Leveraging all-round clues for trustworthy image forgery detection and localization", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Vancouver, Canada, June 2023. arXiv:2212.10957
  33. D. Cozzolino, A. Pianese, M. Nießner, and L. Verdoliva, "Audio-Visual Person-of-Interest DeepFake Detection", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop on Media Forensics, Vancouver, Canada, June 2023. arXiv:2204.03083
  34. R. Corvi, D. Cozzolino, G. Poggi, K. Nagano, and L. Verdoliva, "Intriguing properties of synthetic images: from generative adversarial networks to diffusion models", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshop on Media Forensics, Vancouver, Canada, June 2023. arXiv:2304.06408
  35. A. K. S. Yadav, E. R. Bartusiak, K. Bhagtani, and E. J. Delp, "Synthetic speech attribution using self supervised audio spectrogram transformer", Proceedings of the Media Watermarking, Security, and Forensics Conference, IS&T Electronic Imaging Symposium, pp. 372-1-372-11, San Francisco, CA, January 2023. doi: 10.2352/EI.2023.35.4.MWSF-372
  36. Z. Xiang, A. K. S. Yadav, P. Bestagini, S. Tubaro, and E. J. Delp, "MTN: Forensic Analysis of MP4 Container Metadata Using Graph Neural Networks", Proceedings of the IEEE/CVF Computer Vision and Pattern Recognition Workshops, pp. 963-972, Vancouver, Canada, June 2023. link
  37. Z. Xiang, A. K. S. Yadav, S. Tubaro, P. Bestagini, and E. J. Delp, "Extracting Efficient Spectrograms From MP3 Compressed Speech Signals for Synthetic Speech Detection", Proceedings of the ACM Workshop on Information Hiding and Multimedia Security, pp. 163-168, Chicago, USA, June 2023. doi: 10.1145/3577163.3595104
  38. A. K. S. Yadav, K. Bhagtani, Z. Xiang, P. Bestagini, S. Tubaro, and E. J. Delp, "DSVAE: Disentangled Representation Learning for Synthetic Speech Detection", Proceedings of the International Conference on Machine Learning and Applications, Jacksonville, Florida, USA, December 2023. arXiv:2304.03323
  39. A. K. S. Yadav, Z. Xiang, E. R. Bartusiak, P. Bestagini, S. Tubaro, and E. J. Delp, "ASSD: Synthetic Speech Detection in the AAC Compressed Domain", Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 1-5, Rhodes Island, Greece, June 2023. doi: 10.1109/ICASSP49357.2023.10095043
  40. K. Bhagtani, E. R. Bartusiak, A. K. S. Yadav, P. Bestagini, and E. J. Delp, "Synthesized Speech Attribution Using The Patchout Spectrogram Attribution Transformer", Proceedings of the ACM Workshop on Information Hiding and Multimedia Security, pp. 157-162, Chicago, USA, June 2023. doi: 10.1145/3577163.3595112
  41. A. Ferreira, and M. Barni, "Attacking and Defending Printer Source Attribution Classifiers in the Physical Domain", Proceedings of the Workshop on MultiMedia FORensics in the WILD, Montreal, Canada, August 2022. doi: 10.5281/zenodo.6899744
  42. D. Lin, B. Tondi, B. Li, and M. Barni, "Exploiting temporal information to prevent the transferability of adversarial examples against deep fake detectors", Proceedings of the IEEE International Joint Conference on Biometrics, Abu Dhabi, United Arab Emirates, October 2022. doi: 10.1109/IJCB54206.2022.10007959
  43. O. Alamayreh, G. M. Dimitri, J. Wang, B. Tondi, and M. Barni, "Which Country is This Picture From? New Data and Methods For Dnn-Based Country Recognition", Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Rhodes Island, Greece, June 2023. doi: 10.1109/ICASSP49357.2023.10094908
  44. J. Wang, B. Tondi, and M. Barni, "Classification of Synthetic Facial Attributes by Means of Hybrid Classification/Localization Patch-Based Analysis", Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Rhodes Island, Greece, June 2023. doi: 10.1109/ICASSP49357.2023.10094861
  45. J. Wang, O. Alamayreh, B. Tondi, and M. Barni, "Open Set Classification of GAN-based Image Manipulations via a ViT-based Hybrid Architecture", Proceedings of the IEEE/CVF Computer Vision and Pattern Recognition Workshop on Media Forensics, Vancouver, Canada, June 2023. doi: 10.48550/arXiv.2304.05212
  46. Z. Xiang, P. Bestagini, S. Tubaro, and E. J. Delp, "H4VDM: H.264 Video Device Matching", Proceedings of the Pattern Recognition, Computer Vision and Image Processing, International Workshops and Challenges, pp. 300-320, Springer, Cham, August 2023. doi: 10.1007/978-3-031-37742-6_24
  47. A. K. S. Yadav, Z. Xiang, K. Bhagtani, P. Bestagini, S. Tubaro, and E. J. Delp, "PS3DT: Synthetic Speech Detection Using Patched Spectrogram Transformer", Proceedings of the International Conference on Machine Learning and Applications, Jacksonville, Florida, USA, December 2023.
  48. K. Bhagtani, A. K. S. Yadav, Z. Xiang, P. Bestagini, and E. J. Delp, "FGSSAT : Unsupervised Fine-Grain Attribution of Unknown Speech Synthesizers Using Transformer Networks", Proceedings of the IEEE Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, California, October 2023.
  49. E. R. Bartusiak, K. Bhagtani, A. K. S. Yadav, and E. J. Delp, "Transformer Fusion for Synthesized Speech Detection", Proceedings of the IEEE Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, California, October 2023.

Book Chapters

  1. H. Hao, E. R. Bartusiak, D. Güera, D. Mas, S. Baireddy, Z. Xiang, S. K. Yarlagadda, R. Shao, J. Horváth, J. Yang, F. Zhu, and E. J. Delp, “Deepfake Detection Using Multiple Data Modalities”, in Handbook of Digital Face Manipulation and Detection - From DeepFakes to Morphing Attacks, Series on Advances in Computer Vision and Pattern Recognition, 1st ed., Springer, March 2022. doi: 10.1007/978-3-030-87664-7_11.
  2. M. Yankoski, W. Theisen, E. Verdeja and W. Scheirer, "Artificial Intelligence for Peace: An Early Warning System for Mass Violence", in Towards an International Political Economy of Artificial Intelligence, International Political Economy Series, 2021. doi: 10.1007/978-3-030-74420-5_7

Press Releases

  • A Purdue University press release of our work on Deepfake detection
  • A Purdue University press release of our work on the SemaFor program
  • An article on Purdue Exponent featuring the VIPER lab students' comments on Deepfake

This material is based on research sponsored by the Defense Advanced Research Projects Agency (DARPA) and the Air Force Research Laboratory (AFRL) under agreement number FA8750-20-2-1004. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DARPA and AFRL or the U.S. Government.