Face swapping refers to the task of changing the appearance of a face appearing in an image by replacing it with the appearance of a face taken from another image, in an effort to produce an authentic-looking result. We describe a method for face swapping that does not require training on faces being swapped and can be easily applied even when face images are unconstrained and arbitrarily paired. Our method offers the following contributions: (a) Instead of tailoring systems for face segmentation, as others previously proposed, we show that a standard fully convolutional network (FCN) can achieve remarkably fast and accurate segmentation, provided that it is trained on a rich enough example set. For this purpose, we describe novel data collection and generation routines which provide challenging segmented face examples. (b) We use our segmentations for robust face swapping under unprecedented conditions, without requiring subject-specific data or training. (c) Unlike previous work, our swapping is robust enough to allow for extensive quantitative tests. To this end, we use the Labeled Faces in the Wild (LFW) benchmark and measure how intra- and inter-subject face swapping affect face recognition. We show that intra-subject swapped faces remain as recognizable as their sources, testifying to the effectiveness of our method. In line with established perceptual studies, we show that better face swapping produces less recognizable inter-subject results (see, e.g., Fig. 2.1). This is the first time this effect was quantitatively demonstrated by a machine vision method. Some of the material in this chapter previously appeared in [47].

Face Segmentation, Face Swapping, and How They Impact Face Recognition / Nirkin, Y.; Masi, I.; Tuan Tran, A.; Hassner, T.; Medioni, G.. - (2021), pp. 21-43. - ADVANCES IN COMPUTER VISION AND PATTERN RECOGNITION. [10.1007/978-3-030-74697-1_2].

Face Segmentation, Face Swapping, and How They Impact Face Recognition

Masi I.
Secondo
;
2021

Abstract

Face swapping refers to the task of changing the appearance of a face appearing in an image by replacing it with the appearance of a face taken from another image, in an effort to produce an authentic-looking result. We describe a method for face swapping that does not require training on faces being swapped and can be easily applied even when face images are unconstrained and arbitrarily paired. Our method offers the following contributions: (a) Instead of tailoring systems for face segmentation, as others previously proposed, we show that a standard fully convolutional network (FCN) can achieve remarkably fast and accurate segmentation, provided that it is trained on a rich enough example set. For this purpose, we describe novel data collection and generation routines which provide challenging segmented face examples. (b) We use our segmentations for robust face swapping under unprecedented conditions, without requiring subject-specific data or training. (c) Unlike previous work, our swapping is robust enough to allow for extensive quantitative tests. To this end, we use the Labeled Faces in the Wild (LFW) benchmark and measure how intra- and inter-subject face swapping affect face recognition. We show that intra-subject swapped faces remain as recognizable as their sources, testifying to the effectiveness of our method. In line with established perceptual studies, we show that better face swapping produces less recognizable inter-subject results (see, e.g., Fig. 2.1). This is the first time this effect was quantitatively demonstrated by a machine vision method. Some of the material in this chapter previously appeared in [47].
Advances in Computer Vision and Pattern Recognition
978-3-030-74696-4
978-3-030-74697-1
face recognition, deep learning, face analysis
02 Pubblicazione su volume::02a Capitolo o Articolo
Face Segmentation, Face Swapping, and How They Impact Face Recognition / Nirkin, Y.; Masi, I.; Tuan Tran, A.; Hassner, T.; Medioni, G.. - (2021), pp. 21-43. - ADVANCES IN COMPUTER VISION AND PATTERN RECOGNITION. [10.1007/978-3-030-74697-1_2].
File allegati a questo prodotto
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11573/1569486
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 1
  • ???jsp.display-item.citation.isi??? ND
social impact