1. Proceedings
  2. I3M
  3. 2022
  4. EMSS
  5. 10.46354/i3m.2022.emss.028

Strategies for Semi-Automated Registration of Historic Aerial Photographs Utilizing Street and Roof Segmentations as Durable Landmarks

  • a Gerald A. Zwettler ,
  • b  Yuta Ono,
  • c Michael Stradner,
  • d  Christoph Praschl
  • a,c,d Research Group Advanced Information Systems and Technology, Research and Development Department, University of Applied Sciences Upper Austria
  • b Graduate School of Software and Information Science, Iwate Prefectural University Japan
  • a Department of Software Engineering, School of Informatics, Communications and Media, University of Applied
    Sciences Upper Austria
Cite as
Zwettler G., Ono Y., Stradner M.and Praschl C. (2022).,Strategies for Semi-Automated Registration of Historic Aerial Photographs Utilizing Street and Roof Segmentations as Durable Landmarks. Proceedings of the 34th European Modeling & Simulation Symposium (EMSS 2022). , 028 . DOI: https://doi.org/10.46354/i3m.2022.emss.028

Abstract

Historical and current aerial photographs are only of great value if the geolocation or address of the photographed areas is also available. In Western Europe, especially Austria, Germany and Czech Republic, there is a market for the sale of aerial photographs of one’s own private residential building. Automated geolocation is a feasible way to enable the sales agents to assign the addresses for the sale more quickly. In the course of this research work, a process chain is modeled that allows the assignment of aerial photographs to residential addresses using machine vision. After model-based rectifying the aerial images to compensate for perspective distortions, larger image blocks get assembled using image stitching. The assignment to a 2D reference map, such as satellite imagery via Google Maps, is done by applying a U-Net CNN after extracting durable image features such as roads or buildings. The mapping of aerial imagery to two-dimensional cartography is either automated via registration approaches or based on manually placed corresponding landmarks and homography. Test runs on imagery between the years 1969 and 2020 show that the labor-intensive process of geolocation of aerial imagery can be solved by the proposed process model in a hybrid way.

References

  1. Abadi, M. et al. (2016). Tensorflow: Large-scale machine learning on heterogeneous distributed systems. CoRR, abs/1603.04467
  2. Aicardi, I., Nex, F., Gerke, M., and LINGUA, A. (2016). An image-based approach for the co-registration of multitemporal uav image datasets. Remote Sensing, 9
  3. Alcantarilla, P. F., Bartoli, A., and Davison, A. J. (2012). Kaze features. In Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., and Schmid, C., editors, Computer Vision – ECCV 2012, pages 214–227. Springer Berlin Heidelberg.
  4. Allison, D. and Muller, J. (1993). An automated system for sub-pixel correction and geocoding of multi-spectral and multi-look aerial imagery. Int. Archives of Photogrammetry and Remote Sensing.
  5. Arun, K. S., Huang, T. S., and Blostein, S. D. (1987). Leastsquares fitting of two 3-d point sets. IEEE Trans. on Pat.
    Analysis and Machine Intelligence, PAMI-9:698–700.
  6. Barrow, H. G., Tenenbaum, J. M., Bolles, R. C., and Wolf, H. C. (1977). Parametric correspondence and chamfer matching: Two new techniques for image matching. In IJCAI, pages 659–663.
  7. Bianco, S., Ciocca, G., and Marelli, D. (2018). Evaluating  the performance of structure from motion pipelines. Journal of Imaging, 4:98
  8. Bradski, G. (2000). The OpenCV Library. Dr. Dobb’s Journal of Software Tools.
  9. Brown, M. and Lowe, D. (2007). Automatic panoramic image stitching using invariant features. International Journal of Computer Vision, 74:59–73.
  10. Cheng, K.-S., Yeh, H.-C., and Tsai, C.-H. (2000). An anisotropic spatial modeling approach for remote sensing image rectification. Remote Sensing of Environment, 73(1):46–54
  11. Chmielewski, L. (2004). Choice of the hough transform for image registration. In Proc. SPIE, volume 5505.
  12. Hajnal, J. and Hill, D. (2001). Medical Image Registration. Biomedical Engineering. CRC Press.
  13. Harris, C. and Stephens, M. (1988). A combined corner and edge detector. In In Proc. of Fourth Alvey Vision Conference, pages 147–151.
  14. Henry, C., Azimi, S., and Merkle, N. (2018). Road segmentation in sar satellite images with deep fully convolutional neural networks. IEEE Geoscience and Remote Sensing Letters.
  15. Jaimes, B. and Castro, C. (2018). Perspective correction in aerial images. 10.13140/RG.2.2.34885.29926.
  16. Liu, Z., Chen, B., and Zhang, A. (2020). Building segmentation from satellite imagery using u-net with resnet encoder. In 2020 5th Int. Conf. on Mech., Control and Comp.rEng. (ICMCCE), pages 1967–1971.
  17. Long, T., Jiao, W., He, G., Zhang, Z., Cheng, B., and Wang, W. (2015). A generic framework for image rectification using multiple types of feature. ISPRS Journal of Photogrammetry and Remote Sensing, 102:161–171.
  18. Lowe, D. G. (1999). Object recognition from local scaleinvariant features. In Proceedings of the International
    Conference on Computer Vision-Volume 2 - Volume 2, ICCV ’99, page 1150, USA. IEEE Computer Society
  19. Magalhães, M. (2022). Accurate image alignment and
    registration using openc
  20. Mann, S. and Picard, R. W. (1994). Virtual bellows: constructing high quality stills from video. Proc. of 1st Int.
    Conf. on Image Processing, 1:363–367 vol.1.
  21. Noble, F. K. (2016). Comparison of opencv’s feature detectors and feature matchers. In 2016 23rd Int. Conf. on
    Mechatronics and Machine Vision in Practice.
  22. Pai, M. M., Mehrotra, V., Aiyar, S., Verma, U., and Pai, R. M. (2019). Automatic segmentation of river and land in sar
    images: A deep learning approach. In 2019 IEEE AIKE, pages 15–20
  23. Popescu, G., Balota, L., and Iordan, D. (2015). Section photogrammetry and remote sensing direct georeferencing application of aerial photogrammetry using a gnss/imu/sensor system. In In Proc. of the SGEM Conf., Book 2 Vol.1, pages 1043–1050.
  24. Praschl, C., Stradner, M., Ono, Y., and Zwettler, G. A. (2022). towards an automated system for reverse geocoding of aerial photographs". In Proc. of the 30th Int. Conf. WSCG2022
  25. Ronneberger, O., Fischer, P., and Brox, T. (2015). U-net: onvolutional networks for biomedical image segmentation. CoRR, abs/1505.04597
  26. Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011). Orb: An efficient alternative to sift or surf. In 2011 Int.
    Conf. on Computer Vision, pages 2564–2571.
  27. Seim, H., Kainmueller, D., Heller, M., Zachow, S., and Hege, H.-C. (2009). Automatic extraction of anatomical landmarks from medical image data: An evaluation of different methods. In 2009 IEEE Int. Symp. on Biomedical Imaging: From Nano to Macro, pages 538–541.
  28. Sonka, M. and Fitzpatrick, J. M. (2000). "Handbook of Medical Imaging, Volume 2. Medical Image Processing and
    Analysis". SPIE - Reprint edition (June 14, 2000).
  29. Szeliski, R. (2006). Image Alignment and Stitching, page 273–292. Springer US, Boston, MA.
  30. Wang, Y., Wang, L., Hu, Y. H., and Qiu, J. (2019). Railnet: A segmentation network for railroad detection. IEEE
    Access, 7:143772–143779.
  31. Weiss, K., Khoshgoftaar, T., and Wang, D. (2016). A survey of transfer learning. Journal of Big Data, 3.
  32. Westoby, M., Brasington, J., Glasser, N., Hambrey, M., and Reynolds, J. (2012). ‘structure-from-motion’ photogrammetry: A low-cost, effective tool for geoscience applications. Geomorphology, 179:300–314.
  33. Zheng, Q., Sharf, A., Tagliasacchi, A., Chen, B., Zhang, H., Sheffer, A., and Cohen-Or, D. (2010). Consensus
    skeleton for non-rigid space-time registration. Comput. Graph. Forum, 29:635–644
  34. Zwettler, G., Holmes III, D., and Backfrieder, W. (2020). Strategies for training deep learning models in medical
    domains with small reference datasets. Journal of WSCG, 28(1-2):37–46.