Conceitos essenciais
A lightweight and affordable motion capture method that utilizes two smartwatches and a head-mounted camera, enabling 3D full-body motion capture in diverse environments.
Resumo
The paper presents a novel motion capture method that uses two smartwatches on the wrists and a head-mounted camera to reconstruct 3D full-body human motion. This approach is more cost-effective and convenient compared to existing methods that require six or more expert-level IMU devices.
The key ideas are:
- Integrating 6D head poses obtained from the head-mounted camera to overcome the extreme sparsity and ambiguities of sensor inputs.
- Proposing an algorithm to track and update floor level changes to define head poses, coupled with a multi-stage Transformer-based regression module.
- Leveraging visual cues from the egocentric images to further enhance the motion capture quality and reduce ambiguities.
- Exploring multi-person scenarios where the visual signals among individuals can be shared to provide additional cues for motion capture.
The method is demonstrated on various challenging scenarios, including complex outdoor environments and everyday motions involving object interactions and social interactions.
Estatísticas
"We present a lightweight and affordable motion capture method based on two smartwatches and a head-mounted camera."
"Our method can make wearable motion capture accessible to everyone, enabling 3D full-body motion capture in diverse environments."
"To enable capture in expansive indoor and outdoor scenes, we propose an algorithm to track and update floor level changes to define head poses, coupled with a multi-stage Transformer-based regression module."
"We also introduce novel strategies leveraging visual cues of egocentric images to further enhance the motion capture quality while reducing ambiguities."
Citações
"Our method can make wearable motion capture accessible to everyone, given the widespread availability and popularity of smartwatches, as well as the existence of action cameras and camera glasses."
"Going beyond the traditional optical motion capture methods, primarily feasible in well-set lab environments with limited scope and accessibility, there have been explorations to capture human motions with wearable sensors."
"In contrast to VR settings (e.g., HMDs) where head poses are given in a fixed world coordinate in a small indoor environment, it is not trivial to define head poses in expansive outdoor settings."