Learning Dexterous Manipulation Skills from Human Videos

A new framework called ViViDex is proposed to learn vision-based dexterous manipulation policies from human videos, which consists of three modules: reference trajectory extraction, trajectory guided state-based policy learning, and unified vision-based policy learning.