Adversarially-Guided 3D Shape Deformation via Differentiable Rendering and 2D Supervision

Recovering 3D geometry from 2D observations is a fundamental challenge in computer vision, with applications in animation, virtual reality, and robotics. Recent advances in differentiable rendering have enabled gradient-based optimization of 3D shapes using only image supervision. In this work, we propose a novel adversarial framework that enhances 3D mesh deformation by integrating a differentiable renderer into a Generative Adversarial Network (GAN). The generator deforms an initial mesh and optimizes textures to match 2D supervision from target images, while the discriminator-featuring dense connections and self-attention-learns to distinguish between real and synthesized renderings. Our method improves upon baseline differentiable renderers both quantitatively and qualitatively, achieving lower Chamfer distance and higher Intersection over Union (IoU) across a variety of object categories. The results demonstrate that adversarial training effectively guides mesh deformation, producing reconstructions that are more accurate and visually consistent with target images.