The mass adoption of diffusion models has shown that artificial intelligence (AI) systems can be used to easily generate realistic images. The spread of these technologies paves the way to previously unimaginable creative uses while also raising the possibility of malicious applications. In this work, we propose a critical analysis of the overall pipeline, i.e., from creating realistic human faces with Stable Diffusion v1.5 [1] to recognizing fake ones. We first propose an analysis of the prompts that allow the generation of extremely realistic faces with a human-in-the-loop approach. Our objective is to identify the text prompts that drive the image generation process to obtain realistic photos that resemble everyday portraits captured with any camera. Next, we study how complex it is to recognize these fake contents for both AI-based models and non-expert humans. We conclude that similar to other deepzfake creation techniques, despite some limitations in generalization across different datasets, it is possible to use AI to recognize these contents more accurately than non-expert humans would.

On the use of Stable Diffusion for creating realistic faces: from generation to detection / Papa, Lorenzo; Faiella, Lorenzo; Corvitto, Luca; Maiano, Luca; Amerini, Irene. - (2023). (Intervento presentato al convegno 11th International Workshop on Biometrics and Forensics, IWBF 2023 tenutosi a Barcelona; Spain) [10.1109/IWBF57495.2023.10156981].

On the use of Stable Diffusion for creating realistic faces: from generation to detection

Papa, Lorenzo
;
Corvitto, Luca
;
Maiano, Luca
;
Amerini, Irene
2023

Abstract

The mass adoption of diffusion models has shown that artificial intelligence (AI) systems can be used to easily generate realistic images. The spread of these technologies paves the way to previously unimaginable creative uses while also raising the possibility of malicious applications. In this work, we propose a critical analysis of the overall pipeline, i.e., from creating realistic human faces with Stable Diffusion v1.5 [1] to recognizing fake ones. We first propose an analysis of the prompts that allow the generation of extremely realistic faces with a human-in-the-loop approach. Our objective is to identify the text prompts that drive the image generation process to obtain realistic photos that resemble everyday portraits captured with any camera. Next, we study how complex it is to recognize these fake contents for both AI-based models and non-expert humans. We conclude that similar to other deepzfake creation techniques, despite some limitations in generalization across different datasets, it is possible to use AI to recognize these contents more accurately than non-expert humans would.
2023
11th International Workshop on Biometrics and Forensics, IWBF 2023
Computer vision; Deepfake detection; Diffusion models; Prompt engineering; Security
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
On the use of Stable Diffusion for creating realistic faces: from generation to detection / Papa, Lorenzo; Faiella, Lorenzo; Corvitto, Luca; Maiano, Luca; Amerini, Irene. - (2023). (Intervento presentato al convegno 11th International Workshop on Biometrics and Forensics, IWBF 2023 tenutosi a Barcelona; Spain) [10.1109/IWBF57495.2023.10156981].
File allegati a questo prodotto
File Dimensione Formato  
Papa_On-the-use_2023.pdf

solo gestori archivio

Tipologia: Versione editoriale (versione pubblicata con il layout dell'editore)
Licenza: Tutti i diritti riservati (All rights reserved)
Dimensione 1.36 MB
Formato Adobe PDF
1.36 MB Adobe PDF   Contatta l'autore

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1693691
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 6
  • ???jsp.display-item.citation.isi??? 2
social impact