Benchmarks
3D Camera Re-Localization
Camera Re-Localization is about estimating the precise position and orientation from which a given image was taken with respect to a known 3D scene.
Evaluation and MetricsThe following table lists the benchmark results for the 3D Camera Re-Localization scenario. The score is 1 + DCRE (0.05) - Outlier.
| Method | Info | Score | DCRE (0.05) | DCRE (0.15) | Pose (0.05m, 5°) | Outlier (0.5) | NaN or N/A |
|---|---|---|---|---|---|---|---|
 | |  | | | | ||
| MegLoc | 1.836 | 0.889 | 0.903 | 0.885 | 0.053 | 0.000 | |
| OSpace | 1.397 | 0.504 | 0.730 | 0.173 | 0.107 | 0.062 | |
| DFM | 1.424 | 0.505 | 0.706 | 0.171 | 0.080 | 0.129 | |
| NeuralRouting | 1.441 | 0.538 | 0.615 | 0.358 | 0.097 | 0.227 | |
| Siyan Dong*, Qingnan Fan*, He Wang, Ji Shi, Li Yi, Thomas Funkhouser, Baoquan Chen, Leonidas Guibas: Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments. CVPR 2021 | |||||||
| KAPTURE-R2D2-FUSION |  | 1.338 | 0.447 | 0.612 | 0.176 | 0.109 | 0.000 |
| Martin Humenberger, Yohann Cabon, Nicolas Guerin, Julien Morat, Jérôme Revaud, Philippe Rerole, Noé Pion, Cesar de Souza, Vincent Leroy, Gabriela Csurka: Robust Image Retrieval-based Visual Localization using Kapture. | |||||||
| od_loc | 1.191 | 0.376 | 0.585 | 0.123 | 0.185 | 0.076 | |
| test_1000 | 1.463 | 0.555 | 0.579 | 0.550 | 0.092 | 0.001 | |
| GATs-Loc | 1.401 | 0.533 | 0.579 | 0.506 | 0.132 | 0.196 | |
| D2-Net | | 1.247 | 0.392 | 0.521 | 0.155 | 0.144 | 0.014 |
| Dusmanu, M., Rocco, I., Pajdla, T., Pollefeys, M., Sivic, J., Torii, A., Sattler, T.: D2-Net: A Trainable CNN for Joint Detection and Description of Local Features. CVPR 2019 | |||||||
| Grove v2 | 1.162 | 0.416 | 0.488 | 0.274 | 0.254 | 0.162 | |
| Cavallari*, T., Golodetz*, S., Lord*, N.A., Valentin*, J., Prisacariu, V.A., Stefano, L.D., Torr, P.H.S.: Real-Time RGB-D Camera Pose Estimation in Novel Scenes using a Relocalisation Cascade. TPAMI 2019 | |||||||
| Grove | 1.240 | 0.342 | 0.392 | 0.230 | 0.102 | 0.452 | |
| Cavallari, T., Golodetz, S., Lord, N.A., Valentin, J.P.C., di Stefano, L., Torr, P.H.S.: On-the-Fly Adaptation of Regression Forests for Online Camera Relocalisation. CVPR 2017 | |||||||
| TransAPR | 0.712 | 0.038 | 0.303 | 0.001 | 0.325 | 0.000 | |
| HF-Net (trained) | | 0.789 | 0.192 | 0.300 | 0.073 | 0.403 | 0.000 |
| Sarlin, P.E., Cadena, C., Siegwart, R., Dymczyk, M.: From Coarse to Fine: Robust Hierarchical Localization at Large Scale. CVPR 2019 | |||||||
| Active Search | | 1.166 | 0.185 | 0.250 | 0.070 | 0.019 | 0.690 |
| Sattler, T., Leibe, B., Kobbelt, L.: Efficient & Effective Prioritized Matching for Large-Scale Image-Based Localization. TPAMI 2017 | |||||||
| NetVLAD | 0.575 | 0.007 | 0.137 | 0.000 | 0.431 | 0.000 | |
| Arandjelovi´c, R., Gronat, P., Torii, A., Pajdla, T., Sivic, J.: NetVLAD: CNN architecture for weakly supervised place recognition. CVPR 2016 | |||||||
| DenseVLAD | 0.507 | 0.008 | 0.136 | 0.000 | 0.501 | 0.006 | |
| Torii, A., Arandjelovi´c, R., Sivic, J., Okutomi, M., Pajdla, T.: 24/7 place recognition by view synthesis. CVPR 2015 | |||||||
| HF-Net | | 0.373 | 0.064 | 0.103 | 0.018 | 0.690 | 0.000 |
| Sarlin, P.E., Cadena, C., Siegwart, R., Dymczyk, M.: From Coarse to Fine: Robust Hierarchical Localization at Large Scale. CVPR 2019 | |||||||
| AtLoc+ | 0.423 | 0.006 | 0.088 | 0.000 | 0.582 | 0.000 | |
| MapNet | 0.344 | 0.004 | 0.067 | 0.000 | 0.660 | 0.000 | |
| PoseNet+logq | 0.374 | 0.002 | 0.064 | 0.000 | 0.628 | 0.000 | |