HoloNav
An augmented reality based indoor navigation application.
This article describes the motivation behind the idea of HoloNav along with supporting articles hyperlinked. The code for this application is opensource and can be found here in this repo: HoloNav
Abstract
As our indoor environments are getting complex, larger and thus difficult to navigate, e.g. airports, large warehouses, it becomes seemingly difficult to navigate these spaces in a limited time frame. HoloNav, an AR-based indoor navigation application utilizes onboard cameras and spatial sensors to perform spatial mapping and help an end-user navigate a dynamically registered floor with the help of progenerated 3D mesh map with geo-anchors. The objective of this article is to discuss existing work, introduce HoloNav, discuss the implementation, a comparison of how HoloNav fares with existing techniques and concludes with what HoloNav was able to achieve.
Introduction
Indoor navigation is a problem that currently is solved with the help of beacons and localization hardware present in a receiving device to locate itself. The problem gets harder when it comes to making sure each device irrespective of its capacity to receive signals can receive signals from an external source to locate itself and any changes to the environment will require changes to the pre-registered floor map as well. The most commonly available device — a mobile phone which is equipped with GPS sensors helps it to localize itself well in a larger outdoor environment setting. However, relying on the same technology indoors presents new challenges as the signal weakens while indoors.
The idea of using an application running on a device that is capable of simultaneously localizing and mapping itself without relying on any external source mitigates issues such as depending on other devices to provide data to localize such as GPS, iBeacons and the ability to generate the map on the go mitigates the requirement of a pre-registered floor plan.
Related Work
The idea of using AR for navigation is a pretty well-researched and mature field, however, the main reason for it not being mainstream has been its reliance on certain external sensors for data. In this section, we will be looking at two major existing commercial applications of augmented reality for navigation.
- Live View in Google Maps
Live View is a great example of how augmented reality can help in navigation especially if one is visiting and exploring a new place where the local language of the communication might not be a known to the user. With Live View, the user can use augmented reality to better understand the surroundings with arrows and directions placed in real-world to guide them in a new place. The core idea is to rely on GPS to approximate the user’s position however there are cases where ambiguity arises because of dense urban environments thus making it hard to pinpoint geographic location, or how the device may be held and to mitigate this Google contributes with their virtual position system to help accurately localize the device, i.e. Google uses Street View data to update user’s position in an AR-based outdoor navigation session with buildings as reference points which Google calls as global localization. With this localization data, Google uses their ARCore to bring augmented reality experience to users while navigating dense outdoor environments. The idea to rely on surroundings and fusing the data with Street View helps in accurately localizing the device, however, it is very unlikely that the same technique can be used for indoor localization as there are so many variable factors involved and also large data which can change over a short period. - AR Indoor Navigation by MobiDev
MobiDev has made contributions to indoor navigation with their simplified visual position system where AR Markers are used to localize a device in a pre-saved indoor floor map of an office, mall or a store. The prerequisite in this approach remains the stage where a floor map of the required indoor space must be generated before including the furniture, or other additions that may not be included in the initial construction floor plan. Thus, it has an additional overhead for each new location or changes made to a floor plan in a store or office environment. The second challenge in this approach is the ability to auto-detect elevation changes as most offices consist of multiple floors which can be mitigated only by another AR marker. The drawbacks include the ability to change the markers without prior intimation and thus leading to corruption of the map or misguided users.
HoloNav addresses two important concerns — localizing inside an indoor space automatically without any markers and the ability to navigate inside an indoor space without any need for external sensors.
Implementation Details
The implementation takes advantage of the existing API-level access provided by HoloToolkit. The application itself was developed in Unity with C# making API-level calls to sensors in HoloLens. The application requires spatial perception, webcam, microphone and internet client capabilities enabled. The development environment for first-gen HoloLens can be configured as described in this article. The core application development can be divided into two major stages. The first stage is to develop the application to access sensors in HoloLens and spatially mapping its surroundings and thus generate a 3D mesh map which can be exported using the local portal in HoloLens which then can be imported in Unity, however, the step to import a pre-generated spatial map is completely optional as the application can dynamically calibrate a new map on the go before placement of the anchors. The second stage is to integrate the ability for the application to introduce spatial anchors which are activated using voice commands. The voice command feature relies on Windows Voice API. These anchors are anchored with WorldAnchorStore for them to retain their last saved location whenever the application is restarted, HoloNav can retrieve their saved location once HoloNav localizes itself in the current environment. Lastly, anchor names are to be added, which can be done by connecting the device back to development environment running Unity and naming them, however, naming anchors dynamically on the go using HoloLens was considered a cumbersome step and hence not implemented. Once anchors have been given a name, the application can be built and deployed, thus any change in the environment, HoloNav shall remap the environment on each run while anchors can be easily moved around as the environment has changed. However, the naming or addition of a new anchor will require a redeployment, this is by design and can be easily mitigated, if required, with some lower-level API access.
Comparison
HoloNav successfully mitigates issues that current implementations are facing during their deployment, this section shall discuss both advantages and disadvantages of HoloNav. The head-mounted display along with onboard sensors provides an edge for HoloNav to be a completely hands-free implementation allows users to interact with voice commands and does not require any reliance on sensors that depend on the external source such as GPS, beacons signals which tend to get weaker based on the density and surroundings. Thus, HoloNav relies on spatial mapping to map its surroundings and localize itself which makes it self-reliant when it comes to localization with better precision and accuracy. The hands-free experience allows the user to perform various other tasks easily. On the other hand, a hand-held device is more economical and thus easily deployable in a large setting as opposed to an expensive device such as HoloLens but brings in challenges such as reliance on external sensors for localization, reduced accuracy and increased ambiguity to localize. The HoloNav, on the other hand, is limited by hardware to sense the environment beyond a 5-meter radius. Thus, HoloNav has its niche target demographic that can reap the benefits.
Conclusion
HoloNav was tested with 10 geo-anchors with anchors representing labs, study rooms and lounges on the computer science department floor. HoloNav was successfully able to localize, provide the user with options to navigate and able to help the user navigate the floor without any issue. HoloNav is a complete and promising reliable package when it comes to implementing indoor navigation, there is room to stitch multiple floors, include far more anchors and add more anchors on the fly to make HoloNav a completely commercially viable product. HoloNav finds its application in a niche set-up where a hands-free dynamic indoor navigation capability is a requirement.
