1. Proceedings
  2. I3M
  3. 2018
  4. MAS
  5. 10.46354/i3m.2018.mas.020

Towards retooling the Microsoft HoloLens as outdoor AR and MR device

  • a Christoph Praschl , 
  • b Oliver Krauss  ,
  • c Gerald Zwettler  
  • a,b,c Research Group for Advanced Information Systems and Technology (AIST), Research and Development Department, University of Applied Sciences Upper Austria, Softwarepark 11, 4232 Hagenberg, AUSTRIA
  • c School of Informatics, Communications and Media, University of Applied Sciences Upper Austria, Softwarepark 11, 4232 Hagenberg, AUSTRIA
Cite as
Praschl C., Krauss O., Zwettler G. (2018). Towards retooling the Microsoft HoloLens as outdoor AR and MR device. Proceedings of the 17th International Conference on Modeling & Applied Simulation (MAS 2018), pp. 126-135. DOI: https://doi.org/10.46354/i3m.2018.mas.020

Abstract

This research work covers generic approaches for the determination of the outdoor position and orientation of an augmented reality device due to the lack of outdoor capability of depth-sensor based devices currently available on the market. The determination of the orientation is primarily achieved with an attitude heading reference system (AHRS) for a rough estimation. Based on a connected/inbuild video camera the accuracy at minor changes of the orientation is enhanced applying registration to assess the differences in orientation between two video frames, compensating gyroscope drift errors. The position determination is achieved using GPS with a rover- and base station real time kinematic beacon system to achieve enhanced precision. Results show that due to sensor application AR hardware considered for indoor use can be retooled to properly work outdoors, at large distances and even inside running vehicles. Thus, future implementation of applications in various domains is facilitated.

References

  1. Apple, 2018. ARKit. Available from https://developer.apple.com/arkit/ [accessed 9th May 2018].
  2. Aulinas J., Petillot Y., Savli J., Lladó X., 2008. The SLAM problem: a survey. In Proc. of the 2008 Conference on Artificial Intelligence Research and Development, 363-371.
  3. Boger Y., 2014. Positional tracking: "Outside-in" vs. "Inside-out". Available from https://vrguy.blogspot.co.at/2014/08/positional-trackingoutside-in-vs.html [accessed 9th May 2018].
  4. Bostanci, E., Kanwal, N., Ehsan, S., and Clark, A., 2013. User Tracking Methods for Augmented Reality. International Journal of Computer Theory and Engineering Vol. 5: 93-98.
  5. Buschmann, C., Müller, F., and Fischer, S., 2004. Grid-Based Navigation for Autonomous, Mobile Robots. Technical University of Braunschweig, Braunschweig, Germany.
  6. Caruos, M., 2000. Applications of magnetic sensors for low cost compass systems. IEEE 2000 Position Location and Navigation Symposium, 177-184. 13. – 16. March 2000, San Diego, USA.
  7. Ehrenborg, G., and Fransson, H., 2015. Sensor Fusion based Indoor Positioning with iBeacons. Master´s Thesis. Chalmers University of Technology, Gothenburg.
  8. Elmenreich, W., 2002. Sensor Fusion in Time-Triggered Systems. Doctor´s Thesis. Vienna University of Technology.
  9. Emlid, 2018. Reach RTK kit. Available from https://docs.emlid.com/reach/ [accessed 9th May
    2018].
  10. European Communities, 2007. Co-operation Agreement On a Civil Global navigation Satellite System (GNSS) between the European Community and its Member States and Ukraine Overview. Available from https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/243190/7199.pdf [accessed 9th May 2018].
  11. Google, 2018. ARCore Overview. Available from https://developers.google.com/ar/discover/ [accessed 9th May 2018].
  12. Goshtasby, A. Ardeshir, 2012. Image Registration - Principles, Tools and Methods. Springer, London.
  13. Hesse, S., and Schnell, G., 2014. Sensoren für die Prozess- und Fabrikautomation. Translation: Sensors for process and factory automation. Wiesbaden, Germany: Springer Vieweg.
  14. Huawei, 2018. E5573. Available from https://consumer.huawei.com/lk/support/mobilebroadband/e5573/ [accessed 9th May 2018].
  15. ITK, 2018. Insight Segmentation and Registration Toolkit (ITK). Available from https://itk.org/
    [accessed 9th May 2018].
  16. ImageJ, 2018. ImageJ. Available from https://imagej.net/ [accessed 9th May 2018].
  17. Kalman R., 1960. A new Approach to Linear Filtering and Prediction. Journal of Basic Engineering: 35-45.
  18. Keyglove, 2011. Acceleration, Velocity, Position and Dead Reckoning. Available from http://www.keyglove.net/2011/10/12/accelerationvelocity-position-and-dead-reckoning/ [accessed 9th May 2018].
  19. Madgwick S., 2010. An efficient orientation filter for inertial and intertial/magnetic sensor arrays. Available from http://x-io.co.uk/res/doc/madgwick_internal_report.pdf [accessed 9th May 2018].
  20. Maeda, M., Ogawa, T., Kiyokawa, K., and Takemura, H., 2004. Tracking of user position and
    orientation by stereo measurement of infrared markers and orientation sensing. 8th International Symposium on Wearable Computers, 77-84. 31. October – 03 November 2004, Washington D.C., USA.
  21. Melbourne, A., Ridgway, G., and Hawkes, D.J., 2010. Image Similarity Metrics. In Proc. of the SPIE.
  22. Metavision, 2018. Meta 2 Augmented Reality Development Kit. Available from https://metaeu.myshopify.com/products/meta2 [accessed 9th May 2018].
  23. Microsoft, 2018a. Microsoft HoloLens. Available from https://www.microsoft.com/en-us/HoloLens [accessed 9th May 2018].
  24. Microsoft, 2018b. Spatial mapping design. Available from https://docs.microsoft.com/en-us/windows/mixed-reality/spatial-mapping-design [accessed 9th May 2018].
  25. Microsoft, 2018c. What´s a Universal Windows Platform (UWP) app?. Available from https://docs.microsoft.com/en-us/windows/uwp/get-started/universal-application-platform-guide [accessed 9th May 2018].
  26. Nestares, O., Gat, Y., Haussecker, H., and Kozintsev, I., 2010. Video stabilization to a global 3D frame of reference by fusing orientation sensor and image alignment data. In IEEE International Symposium on Mixed and Augmented Reality, Seoul, 2010, pp. 257-258.
  27. Qiu S., Wang Z., Zhao H., Qin K., Li Z., Hu H., 2008. Inertial/magnetic sensors based pedestrian dead reckoning by means of multi-sensor fusion. Information Fusion, Vol. 39: 108-119.
  28. OpenCV, 2018a. OpenCV. Available from https://opencv.org/ [accessed 9th May 2018].
  29. OpenCV, 2018b. Motion Analysis and Object Tracking.Available from https://docs.opencv.org/2.4/modules/imgproc/doc/motion_analysis_and_object_tracking.html [accessed 9th May 2018].
  30. OpenPnP, 2017. OpenCV 3.2.0 (packaged by OpenPnP). Available from https://github.com/ openpnp/opencv [accessed 9th May 2018].
  31. Papagiannakis G., Singh G., Magnenat-Thalmann N., 2008. A survey of mobile and wireless
    technologies for augmented reality systems. Computer Animation and Virtual Worlds, Vol. 19: 3-22.
  32. Pedley M., 2013. Tilt sensing using a three-axis accelerometer. Available from: https://cache. freescale.com/files/sensors/doc/app_note/AN3461. pdf [accessed 9th May 2018]
  33. Polischuk G., Kozlov V., Ilitchov V., Kozlov A., Bartenev V., Kossenko V., Anphimov N.,
    Revnivykh S., Pisarev S., Tyulyakov A., Shebshaevitch B., Basevitch A., Vorokhovsky Y., 2002. The global navigation satellitesystem GLONASS: development and usage in the 21st century. 34th Annual Precise Time and Time Interval (PTTI) Meeting: 151-160, The Institute of Navigation Inc, Manassas, USA.
  34. PTC Inc., 2018a. Vuforia. Available from https://www.vuforia.com/ [accessed 9th May 2018].
  35. PTC Inc., 2018b. Image Targets. Available from https://library.vuforia.com/articles/Training/Image-Target-Guide [accessed 9th May 2018].
  36. Praschl C., 2017. Bildbasierte Orientierung im Outdoorbereich. Translation: Image-based Outdoor Orientation. Bachelor´s Thesis. University of Applied Sciences Upper Austria.
  37. Pivotal Software, 2018. Spring. Available from https://spring.io/ [accessed 9th May 2018].
  38. Sutherland I., 1968. A head-mounted three-dimensional display. Fall Joint Computer Conference, 757-764. 09 – 11 December 1968, Salt Lake City, USA.
  39. Tomaszewski, D., Rapinski J., and Pelc-Mieczkowska R. I., 2017. Concept of AHRS algorithm designed for platform independent IMU attitude alignment. Reports on Geodesy and Geoinformatics, Vol 104: 33-47.
  40. Tsui J., 2005. Fundamentals of Global Positioning System Receivers: A Software Approach. New Jersey, USA: Wiley.
  41. U.S. Air Force, 2008. Global Positioning System Wide Area Augmentation System Performance Standard. Available from https://www.gps.gov/technical/ps/2008-WAAS-performancestandard. pdf [accessed 9th May 2018].
  42. Unity Technologies, 2018a. Unity. Available from https://unity3d.com/ [accessed 9th May 2018].
  43. Unity Technologies, 2018b. Camera.main. Available from https://docs.unity3d.com/ScriptReference/Camera-main.html [accessed 9th May 2018].
  44. Unity Technologies, 2018c. GameObject. Available from https://docs.unity3d.com/ScriptReference/GameObject.html [accessed 9th May 2018].
  45. Wikitude, 2018. Wikitude. Available from https://www.wikitude.com/ [accessed 9th May 2018].
  46. WIT Motion, 2018. Attitude Sensor. Available. from http://www.wit-motion.com/english.php?m=goods&a=details&content_id=96 [accessed 9th May 2018].
  47. x-io Technologies, 2016. Next Generation IMU. Available from http://x-io.co.uk/ngimu/ [accessed 9th May 2018].
  48. x-io Technologies, 2017. NGIMU-Unity-Example. Available from https://github.com/xio Technologies/NGIMU-Unity-Example/blob/master/Simple%20Demo/Assets/Code/UnityHelpe r.cs [accessed 9th May 2018].
  49. Xu H., 2012. Application of GPS-RTK Technology in the Land Change Survey. International Workshop on Information and Electronics Engineering, 3454 – 3459. 10. -11. March 2012, Harbin, China.
  50. Zaidi, M., Rached, T., and Ouni, R., 2010. A new geometric approach to mobile position in wireless LAN reducing complex computations. 5th International Conference on Design & Technology of Integrated Systems in Nanoscale Era, 1-7. 23-25 March 2010, Hammamet, Tunisia.