The paper presents a method to transform a non-differentiable rasterizer into a differentiable one, allowing for the optimization of 3D assets within an existing rendering engine. The key insights are:
Using Stochastic Gradient Estimation to estimate gradients without a differentiable framework. This involves randomly perturbing the scene parameters and computing the gradient based on the difference in the rasterized images.
Estimating gradients on a per-pixel basis rather than the full image. This bounds the dimensionality of the optimization problem and makes the method scalable to scenes with millions of parameters.
The authors implement their method by adding two compute shaders to an existing rasterization engine. The first shader perturbs the scene parameters, the second computes the gradients based on the perturbed and original images. This approach keeps the workflow self-contained, cross-platform, and efficient, while supporting a wide range of primitives such as meshes, textures, volumes, subdivision surfaces, and physically-based materials.
The paper provides a detailed validation of the per-pixel formulation, showing that it significantly outperforms the full-image approach in high-dimensional optimization problems. It also includes a qualitative comparison to the state-of-the-art differentiable rasterizer nvDiffRast, demonstrating that their method can produce similar results while being simpler to implement and integrate into an existing engine.
เป็นภาษาอื่น
จากเนื้อหาต้นฉบับ
arxiv.org
ข้อมูลเชิงลึกที่สำคัญจาก
by Thomas Delio... ที่ arxiv.org 04-16-2024
https://arxiv.org/pdf/2404.09758.pdfสอบถามเพิ่มเติม