Perturb, Attend, Detect and Localize (PADL): Robust Proactive Image Defense

Image manipulation detection has gained significant attention due to the rise of Generative Models (GMs). Passive detection methods often overfit to specific GMs, limiting their effectiveness. Recently, proactive approaches have been introduced to overcome this limitation. However, these methods suffer from two vulnerabilities: i) the manipulation detector is not robust to noise and hence can be easily fooled; ii) they rely on fixed perturbations for image protection, which offers an exploit for malicious attackers, enabling them to evade detection. To overcome these issues, we propose PADL, a novel solution that is able to create image-specific perturbations for protecting images. PADL’s key objective is to provide a secure and adaptive protection mechanism that ensures the authenticity of images by detecting and localizing manipulations, drastically reducing the possibility of reverse engineering. The method consists of two key components: an encoder, which conditions a learnable perturbation on the input image to ensure uniqueness and robustness against attacks, and a decoder, which extracts the perturbation and leverages it for manipulation detection and localization. PADL can detect manipulation of a protected image and pinpoint regions that have undergone alterations. Unlike previous proactive defenses that rely on a finite set of perturbations, PADL’s tailored protection significantly reduces the risk of reverse engineering. Although being trained only on images of faces manipulated with STGAN, PADL generalizes to a range of unseen models with diverse architectural designs, such as StarGANv2, CycleGAN, BlendGAN, DiffAE, StableDiffusion, and StableDiffusionXL and also to unseen data domains. Finally, we propose a novel evaluation protocol that fairly assesses localization performance in relation to detection accuracy, providing a better reflection of real-world scenarios. Future research will aim to extend PADL to work on more challenging scenarios, including video content protection and high-resolution images, ensuring its effectiveness across diverse media formats and real-world applications. The source code will be publicly released.