Core Concepts
AdaFold introduces a model-based feedback-loop framework to optimize cloth folding trajectories by leveraging semantic descriptors and model predictive control.
Abstract
AdaFold presents a novel approach to adapt folding trajectories using feedback-loop manipulation, demonstrating success in optimizing cloth folding across various physical properties and real-world scenarios. The framework integrates semantic descriptors from visual-language models to enhance the particle representation of cloth, showcasing improved performance compared to baselines.
The content discusses the challenges in robotic manipulation of deformable objects like cloth due to state estimation difficulties and dynamics modeling limitations. It highlights recent advancements in learning cloth dynamics through model-based methods but emphasizes the need for feedback-loop manipulation strategies. AdaFold is proposed as a solution that leverages semantic knowledge from pre-trained visual-language models to enhance point cloud representations of cloth for better trajectory optimization.
Experiments validate AdaFold's ability to adapt folding trajectories across different physical properties and variations in real-world scenarios. The framework combines perception modules with data-driven optimization strategies, showcasing the potential of feedback-loop manipulation in robotic tasks. Future work includes extending AdaFold's applications to diverse clothing items and tasks beyond folding.
Stats
"AdaFold extracts a particle-based representation of cloth from RGB-D images."
"Our experiments demonstrate AdaFold’s ability to adapt folding trajectories to cloths with varying physical properties."
"We further propose to use pre-trained VLM to extract semantic descriptors of the upper and bottom layers of the cloth from RGB images."