Deep fake technology paves the way for a new generation of super realistic artificial content. While this opens the door to extraordinary new applications, the malicious use of deepfakes allows for far more realistic disinformation attacks than ever before. In this paper, we start from the intuition that generating fake content introduces possible inconsistencies in the depth of the generated images. This extra information provides valuable spatial and semantic cues that can reveal inconsistencies facial generative methods introduce. To test this idea, we evaluate different strategies for integrating depth information into an RGB detector and we propose an attention mechanism that makes it possible to integrate information from depth effectively. In addition to being more accurate than an RGB model, our Masked Depthfake Network method is +3.2% more robust against common adversarial attacks on average than a typical RGB detector. Furthermore, we show how this technique allows the model to learn more discriminative features than RGB alone.

A guided-based approach for deepfake detection: RGB-depth integration via features fusion / Leporoni, G.; Maiano, L.; Papa, L.; Amerini, I.. - In: PATTERN RECOGNITION LETTERS. - ISSN 0167-8655. - 181:(2024), pp. 99-105. [10.1016/j.patrec.2024.03.025]

A guided-based approach for deepfake detection: RGB-depth integration via features fusion

Maiano L.
;
Papa L.
;
Amerini I.
2024

Abstract

Deep fake technology paves the way for a new generation of super realistic artificial content. While this opens the door to extraordinary new applications, the malicious use of deepfakes allows for far more realistic disinformation attacks than ever before. In this paper, we start from the intuition that generating fake content introduces possible inconsistencies in the depth of the generated images. This extra information provides valuable spatial and semantic cues that can reveal inconsistencies facial generative methods introduce. To test this idea, we evaluate different strategies for integrating depth information into an RGB detector and we propose an attention mechanism that makes it possible to integrate information from depth effectively. In addition to being more accurate than an RGB model, our Masked Depthfake Network method is +3.2% more robust against common adversarial attacks on average than a typical RGB detector. Furthermore, we show how this technique allows the model to learn more discriminative features than RGB alone.
2024
Computer vision; Deepfake; Feature fusion; RGBD
01 Pubblicazione su rivista::01a Articolo in rivista
A guided-based approach for deepfake detection: RGB-depth integration via features fusion / Leporoni, G.; Maiano, L.; Papa, L.; Amerini, I.. - In: PATTERN RECOGNITION LETTERS. - ISSN 0167-8655. - 181:(2024), pp. 99-105. [10.1016/j.patrec.2024.03.025]
File allegati a questo prodotto
File Dimensione Formato  
Leporoni_A-guided-based_2024.pdf

accesso aperto

Note: DOI10.1016/j.patrec.2024.03.025
Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Creative commons
Dimensione 1.02 MB
Formato Adobe PDF
1.02 MB Adobe PDF

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1710641
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 2
  • ???jsp.display-item.citation.isi??? 1
social impact