Estimating Physical Properties of Objects from Language-Embedded 3D Feature Fields

Humans can perceive the physical properties of objects solely through vision by associating visual appearances with grounded knowledge about materials. This paper presents a novel approach to equip computers with similar capabilities by leveraging large language models and vision-language embeddings to estimate a variety of physical properties, such as mass, friction, and hardness, in a zero-shot manner from a collection of object images.