The paper proposes GaitSTR, a method for gait recognition that combines silhouette and skeleton representations. The key insights are:
Silhouettes lack detailed part information when there is overlap between body segments, and are affected by carried objects and clothing. Skeletons provide more accurate part information but are sensitive to occlusions and low-quality images, causing inconsistencies in frame-wise results.
GaitSTR refines the skeleton representation by leveraging the temporal consistency between silhouettes and skeletons. It introduces two-level fusion: internal fusion within skeletons (between joints and bones) and cross-modal correction with temporal guidance from silhouettes.
The internal fusion uses self-correction residual blocks to improve consistency between joints and bones in the skeleton representation. The cross-modal fusion uses silhouette features to predict relative changes for joints and bones, refining the skeleton.
Experiments on four public gait recognition datasets show that the refined skeletons, when combined with silhouettes, outperform other state-of-the-art methods that use skeletons and silhouettes.
לשפה אחרת
מתוכן המקור
arxiv.org
תובנות מפתח מזוקקות מ:
by Wanrong Zhen... ב- arxiv.org 04-04-2024
https://arxiv.org/pdf/2404.02345.pdfשאלות מעמיקות