Enhancing controllability in procedural and non-procedural asset editing

The design of 2D and 3D assets is one of the key components of Computer Graphics applications. Recently, the demand for high-quality assets, like textures and materials, or 3D shapes, has seen impressive growth due to their widespread adoption in industrial design workflows as well as in the movie and video game industries. However, designing an asset is still a consuming operation in terms of time and human effort, in particular when a precise target must be matched. Over the years, various techniques, such as real-world material and model captures or inverse modeling, have been developed to ease this process and to support artists in matching a desired target. This thesis, entitled Enhancing Controllability in Procedural and Non-Procedural Asset Editing, explores the common techniques adopted in asset design and proposes novel approaches to reduce the time and human effort involved in this process, aiming to reach an easier and yet more controllable pipeline for asset editing in both 2D and 3D environments.  We first investigate the procedural asset fields, proposing inverse procedural modeling solutions for 2D vector patterns and 3D implicit shapes defined as differentiable programs and graphs, respectively. Specifically, we propose an example-based parameter estimation tool for 2D procedural vector patterns. By exploiting a differentiable SDF definition of a pattern, our framework enables the parameter estimation from a target image in a gradient descent-based environment.  We also investigate direct manipulation approaches for both 2D procedural vector patterns and 3D procedural implicit shapes, enabling the user to edit assets by clicking and dragging elements at interactive rates directly in the viewport. Finally, we concentrate on non-procedural asset synthesis, specifically focusing on structured texture generation. We propose a method for the expansion of a small user-designed sketch to a large-scale, high-quality, and moreover, tileable content. We exploit previously assessed diffusion pipelines as a backbone and inject structured pattern domain knowledge through a LoRA finetuning process, adopting the Noise Rolling technique to improve quality and ensure tileability. In conclusion, this thesis makes a significant contribution to improving user control in procedural and non-procedural asset editing. It proposes novel methods that could be either integrated into common 2D or 3D modeling software and further expanded to various asset design workflows, making the design process easier for both novice and experienced users.