Example of the ground-to-aerial matching problem. The query ground-view image is matched to the polar transformed aerial image
Nowadays the accurate geo-localization of ground-view images has an important role across domains as diverse as journalism, forensics analysis, transports, and Earth Observation. This work addresses the problem of matching a query groundview image with the corresponding satellite image without GPS data. This is done by comparing the features from a ground-view image and a satellite one, innovatively leveraging the corresponding latter’s segmentation mask through a three-stream Siamese-like network. The proposed method, Semantic Align Net (SAN), focuses on limited Field-of-View (FoV) and ground panorama images (images with a FoV of 360°). The novelty lies in the fusion of satellite images in combination with their segmentation masks, aimed at ensuring that the model can extract useful features and focus on the significant parts of the images. This work shows how SAN through semantic analysis of images improves the performance on the unlabelled CVUSA dataset for all the tested FoVs.
We created a subset of the CVUSA dataset on which we trained and tested SAN model.
| Trained Fov | Tested FoV 360° | Tested FoV 180° | Tested FoV 90° | Tested FoV 70° | ||||||||||||
| r@1 | r@5 | r@10 | r@1% | r@1 | r@5 | r@10 | r@1% | r@1 | r@5 | r@10 | r@1% | r@1 | r@5 | r@10 | r@1% | |
| 360° | 77,07% | 92,14% | 95,62% | 97,97% | 47,63% | 75,30% | 83,43% | 90,88% | 18,65% | 38,92% | 48,26% | 60,00% | 12,28% | 28,04% | 36,84% | 47,40% |
| 180° | 67,67% | 86,95% | 92,46% | 96,12% | 48,49% | 75,53% | 84,06% | 91,24% | 21,08% | 43,21% | 53,50% | 65,91% | 14,67% | 30,84% | 39,82% | 52,55% |
| 90° | 23,07% | 45,96% | 56,75% | 70,16% | 13,72% | 30,97% | 42,80% | 56,88% | 6,23% | 16,43% | 24,20% | 37,07% | 3,79% | 12,10% | 18,87% | 29,75% |
| 70° | 14,67% | 33,18% | 42,93% | 56,52% | 7,27% | 21,40% | 30,43% | 44,20% | 3,43% | 12,19% | 19,82% | 31,24% | 3,02% | 9,75% | 15,44% | 25,82% |
Francesco Pro, Nikolaos Dionelis, Luca Maiano, Bertrand Le Saux, and Irene Amerini, “A semantic segmentation-guided approach for ground-to-aerial im- age matching,” IGARSS, 2024.
@Article{
author = "Francesco Pro and Nikolaos Dionelis and Luca Maiano and Bertrand Le Saux and Irene Amerini",
title = "A Semantic Segmentation-guided Approach for Ground-to-Aerial Image Matching",
journal = "IGARSS",
year = "2024. Also: arXiv:2404.11302"
}