Enhancing 3D Understanding via Unified Representation Learning of RGB Images, Depth Images, and Point Clouds through Differentiable Rendering

The proposed DR-Point framework learns a unified representation space by aligning features from RGB images, depth images, and 3D point clouds through contrastive learning and differentiable rendering, leading to significant improvements in a wide range of 3D understanding tasks.