1. Proceedings
  2. I3M
  3. 2019
  4. DHSS
  5. 10.46354/i3m.2019.dhss.001

Deep learning of virtual-based aerial images: increasing the fidelity of serious games for live training

  • a Dean Reed  ,
  • b Troyle Thomas  ,
  • c Shane Reynolds  ,
  • d Jonathan Hurter  ,
  • e Latika Eifert  
  • abcd Institute for Simulation and Training, University of Central Florida, USA
  • e U.S. Army Futures Command, Combat Capability Development Command-Soldier Center
Cite as
Reed D., Thomas T., Reynolds S., Hurter J., Eifert L. (2019). Deep learning of virtual-based aerial images: increasing the fidelity of serious games for live training. Proceedings of the 9th International Defense and Homeland Security Simulation Workshop (DHSS 2019), pp. 1-9. DOI: https://doi.org/10.46354/i3m.2019.dhss.001
 Download PDF

Abstract

The aim of rapidly reconstructing high-fidelity, Synthetic Natural Environments (SNEs) may benefit from a deep learning algorithm: this paper explores how deep learning on virtual, or synthetic, terrain assets of aerial imagery can support the process of quickly and effectively recreating lifelike SNEs for military training, including serious games. Namely, a deep learning algorithm was trained on small hills, or berms, from a SNE, derived from real-world geospatial data. In turn, the deep learning algorithm’s level of classification was tested. Then, assets learned (i.e., classified) from the deep learning were transferred to a game engine for reconstruction. Ultimately, results suggest that deep learning will support automated population of highfidelity SNEs. Additionally, we identify constraints and possible solutions when utilising the commercial game engine of Unity for dynamic terrain generation.

References

  1. Baker T., 2018. VBS3 Resource. Milgaming website. Available from: https://milgaming.army.mil/VBS3/files/ResourceL ist.aspx [accessed 14 Mar 2019].
  2. Chen Y., Jiang H., Li C., Jia X., Ghamisi P. , 2016. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 54 (10), 6232-6251.
  3. Kemker R., Salvaggio C., Kanan, C., 2018. Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning. ISPRS Journal of Photogrammetry and Remote Sensing, 145, 60-77.
  4. Micro Aerial Projects L.L.C., n. d. V-map AIR 20Hz GNSS receiver technical specifications. Micro Aerial Projects L.L.C.
  5. PEO STRI, n.d. Virtual targets. PEO STRI website. Available from: https://www.peostri.army.mil/virtual-targets [accessed 14 May 2019]
  6. Reed D., Thomas T., Eifert L., Reynolds S., Hurter J., Tucker F., 2018. Leveraging virtual environments to train a deep learning algorithm. Proceedings of the 17th International Conference on Modeling and Applied Simulation (MAS 2018), September 17- 19, Budapest (Hungary).
  7. Spicer R., McAlinden R., Conover D., 2016. Producing usable simulation terrain data from UAS-collected imagery. Proceedings of the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC), pp. 1-13.
  8. Tchou C., 2018. 2018.3 terrain update: Getting started. Unity Blog. Available from:
    https://blogs.unity3d.com/2018/10/10/2018-3- terrain-update-getting-started/ [accessed 10 May 2019]
  9. Thys, S., Van Ranst, W., & Goedemé, T. 2019. Fooling automated surveillance cameras: adversarial patches to attack person detection. arXiv preprintarXiv:1904.08653.
  10. Zhang L., Li Z., Li A., Liu F., 2018. Large-scale urban point cloud labeling and reconstruction. ISPRS Journal of Photogrammetry and Remote Sensing, 138, 86-100.