The authors present a novel learning framework for visual hand dynamics analysis that takes into account the physiological aspects of hand motion. Existing models often produce unnatural motions as they simplify the hand as a joint-actuated system.
To address this, the authors integrate a musculoskeletal system with the MANO parametric hand model to create a new model called MS-MANO. This model emulates the dynamics of muscles and tendons to drive the skeletal system, imposing physiologically realistic constraints on the resulting torque trajectories.
The authors further propose a simulation-in-the-loop pose refinement framework, BioPR, that refines the initial estimated pose through a multi-layer perceptron (MLP) network. BioPR takes the predicted hand pose and velocity as input, estimates the muscle excitation signals, and then uses a simulator to generate a reference pose. This reference pose is then used to refine the initial estimated pose.
The accuracy of MS-MANO is compared with MyoSuite, while the efficacy of BioPR is benchmarked against two large-scale public datasets (DexYCB and OakInk) and two recent state-of-the-art methods (gSDF and Deformer). The results demonstrate that the proposed approach consistently improves the baseline methods both quantitatively and qualitatively.
다른 언어로
소스 콘텐츠 기반
arxiv.org
더 깊은 질문