Kernkonzepte
PRAM proposes a novel visual localization framework that efficiently recognizes landmarks in the 3D map and performs fast semantic-aware registration between 2D keypoints and 3D landmarks for accurate pose estimation.
Zusammenfassung
The paper introduces the Place Recognition Anywhere Model (PRAM) for efficient and accurate visual localization. PRAM consists of two main components: recognition and registration.
Landmark Definition:
- The 3D map is reconstructed using deep local features and graph-based matching.
- Landmarks are defined by hierarchically clustering the 3D points on the ground plane, allowing any place to act as a unique landmark.
- Each landmark has a virtual reference frame that observes the majority of its 3D points.
Sparse Recognition:
- Sparse keypoints extracted from the query image are used as tokens to be fed into a transformer-based deep neural network for landmark recognition.
- The recognition module predicts landmark labels for the keypoints, enabling efficient coarse localization.
- Keypoints without corresponding 3D points are identified and discarded as outliers.
Landmark-wise Registration:
- The recognized landmarks and their 2D keypoints are used for fast semantic-aware 2D-3D matching, avoiding the need for exhaustive 2D-2D matching.
- The 2D-3D matches are used with PnP and RANSAC to estimate the final 6DoF pose of the query image.
Compared to prior methods, PRAM achieves higher accuracy in large-scale scenes and significantly higher time and memory efficiency by discarding global and local descriptors and reducing over 90% storage.
Statistiken
PRAM is 50x smaller and 2.4x faster than previous state-of-the-art hierarchical methods.
PRAM outperforms absolute pose regression (APR) and scene coordinate regression (SCR) methods in terms of accuracy in large-scale scenes.
Zitate
"Humans localize themselves efficiently in known environments by first recognizing landmarks defined on certain objects and their spatial relationships, and then verifying the location by aligning detailed structures of recognized objects with those in the memory."
"PRAM discards global and local descriptors, and reduces over 90% storage. Since PRAM utilizes recognition and landmark-wise verification to replace global reference search and exhaustive matching respectively, it runs 2.4 times faster than prior state-of-the-art approaches."