核心概念
APAP, a novel mesh deformation framework, leverages 2D diffusion priors to produce visually plausible deformations of 2D and 3D triangular meshes while preserving the geometric properties of the original shape.
要約
The paper presents APAP, a novel mesh deformation framework that aims to produce visually plausible deformations of 2D and 3D triangular meshes. The key aspects of the framework are:
Representation of the mesh as a Jacobian field: The mesh is parameterized as a set of per-face Jacobians, which can be updated via gradient-descent optimization.
Incorporation of 2D diffusion priors: APAP leverages a pretrained 2D diffusion model (Stable Diffusion) to extract plausibility priors. The diffusion model is finetuned using LoRA to preserve the identity of the edited mesh.
Two-stage optimization: The framework consists of two stages - the first stage deforms the mesh based on user-specified constraints (handle displacements), while the second stage jointly optimizes the Jacobian field to balance the user constraints and the plausibility priors extracted from the diffusion model.
The paper evaluates APAP on both 3D and 2D mesh deformation tasks, demonstrating qualitative and quantitative improvements over baseline methods that rely solely on geometric priors. The experiments show that APAP can produce more plausible deformations while preserving the identity of the edited objects.