The proliferation of algorithms and commercial tools for the creation of synthetic audio has resulted in a significant increase in the amount of inaccurate information, particularly on social media platforms. As a direct result of this, efforts have been concentrated in recent years on identifying the presence of content of this kind. Despite this, there is still a long way to go until this problem is adequately addressed because of the growing naturalness of fake or synthetic audios. In this study, we proposed different networks configurations: a Custom Convolution Neural Network (cCNN) and two pretrained models (VGG16 and MobileNet) as well as end-to-end models to classify real and fake audios. An extensive experimental analysis was carried out on three classes of audio manipulation of the dataset FoR deepfake audio dataset. Also, we combined such sub-datasets to formulate a combined dataset FoR-combined to enhance the performance of the models. The experimental analysis shows that the proposed cCNN outperforms all the baseline models and other reference works with the highest accuracy of 97.23% on FoR-combined and sets new benchmarks for the datasets.

Deepfakes Audio Detection Leveraging Audio Spectrogram and Convolutional Neural Networks / Wani, T. M.; Amerini, I.. - 14234 LNCS:(2023), pp. 156-167. (Intervento presentato al convegno 22nd International Conference on Image Analysis and Processing – ICIAP 2023. tenutosi a Udine, Italy) [10.1007/978-3-031-43153-1_14].

Deepfakes Audio Detection Leveraging Audio Spectrogram and Convolutional Neural Networks

Wani T. M.
Primo
Writing – Original Draft Preparation
;
Amerini I.
Secondo
Supervision
2023

Abstract

The proliferation of algorithms and commercial tools for the creation of synthetic audio has resulted in a significant increase in the amount of inaccurate information, particularly on social media platforms. As a direct result of this, efforts have been concentrated in recent years on identifying the presence of content of this kind. Despite this, there is still a long way to go until this problem is adequately addressed because of the growing naturalness of fake or synthetic audios. In this study, we proposed different networks configurations: a Custom Convolution Neural Network (cCNN) and two pretrained models (VGG16 and MobileNet) as well as end-to-end models to classify real and fake audios. An extensive experimental analysis was carried out on three classes of audio manipulation of the dataset FoR deepfake audio dataset. Also, we combined such sub-datasets to formulate a combined dataset FoR-combined to enhance the performance of the models. The experimental analysis shows that the proposed cCNN outperforms all the baseline models and other reference works with the highest accuracy of 97.23% on FoR-combined and sets new benchmarks for the datasets.
2023
22nd International Conference on Image Analysis and Processing – ICIAP 2023.
Audio Deepfakes; FoR dataset; CNN; VGG16; MobileNet
04 Pubblicazione in atti di convegno::04b Atto di convegno in volume
Deepfakes Audio Detection Leveraging Audio Spectrogram and Convolutional Neural Networks / Wani, T. M.; Amerini, I.. - 14234 LNCS:(2023), pp. 156-167. (Intervento presentato al convegno 22nd International Conference on Image Analysis and Processing – ICIAP 2023. tenutosi a Udine, Italy) [10.1007/978-3-031-43153-1_14].
File allegati a questo prodotto
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1713910
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 4
  • ???jsp.display-item.citation.isi??? 3
social impact