This study focuses on recognizing hidden object characteristics in robotic manipulation tasks by leveraging a two-phase cross-modal transfer learning approach. The first phase involves training a vision module to observe object characteristics directly, while the second phase uses haptic-audio and motor data for indirect sensing. By transferring the learned latent space from vision to haptic-audio, the model can improve recognition accuracy of shape, position, and orientation of objects within containers. The study demonstrates successful online recognition of trained and untrained objects using a humanoid robot setup. Various experiments and evaluations showcase the effectiveness and potential applicability of the proposed method in enhancing robotic perception and manipulation.
To Another Language
from source content
arxiv.org
Key Insights Distilled From
by Namiko Saito... at arxiv.org 03-19-2024
https://arxiv.org/pdf/2403.10689.pdfDeeper Inquiries