The HoloLens could receive a major cloud-powered boost

To achieve mass-market appeal, an augmented reality headset must be compact and light enough for regular use, which limits the amount of hardware that can fit inside. It’s a restriction that often makes it difficult to perform the complex rendering involved in generating 3D content. Microsoft, however, isn’t about to let that get in the way of its plans for the HoloLens.

The company has applied to patent a mechanism that can offload some of the computational heavy-lifting from its headset to the cloud. The filing doesn’t go into detail about the implementation, but Microsoft presumably plans to use its own Azure infrastructure-as-a-service platform for the backend component. Redmond has a very specific use in mind for the system: streamlining the initial environmental modelling that the HoloLens performs when a user visits a new location.

The process involves creating a 3D representation of the objects in the wearer’s view that is known as a mesh. It’s somewhat similar to the model of a computer-generated character before color, lighting effects and other details are added for realism. Capturing the layout of an environment in this way enables the HoloLens to tailor every piece of content for its specific background. It’s an essential part of rendering objects that interact with the real world and many simpler types of content,  too. Even a task as basic as overlaying a piece of virtual text on a whiteboard requires the headset to identify the size and position of the target surface.

A HoloLens connected to Microsoft’s proposed cloud-based offloading mechanism would approach the task quite differently than its present-day peers. The company doesn’t intend to change how the modelling itself is performed, but maps created through the process will be handled in a new way. Instead of discarding a mesh as soon as a user moves to a different location or shuts down their headset, Redmond plans on having the data sent to its backend cloud environment.

The platform will stitch the map together with uploads from the same area to create what the company describes as a global 3D model. From there, the individual parts will periodically refreshed as new uploads come in to ensure that changes made to the location over time are reflected in the backend representation. Microsoft’s goal is to provide the equivalent of a crowdsourced Google Maps to HoloLens customers.

When entering an area, future iterations of the headset may be able to check if there are any readily-available maps of  the location and simply download them over the web. It’s a much more efficient approach than having than having each HoloLens perform modelling locally. As a result, Microsoft’s proposed system could reduce rendering delays while freeing up computational power that can be used to providers higher quality graphics. Having a headset send data about its surroundings over the data raises obvious privacy concerns, but they’re mitigated by the fact that the HoloLens is designed for business rather than personal use.

The mapping service is the latest in a series of augmented reality technologies that Microsoft has moved to patent in recent weeks. Previously, the company filed the paperwork for a method to reduce processing requirements by applying pre-computed lighting effects to objects instead of calculating the ideal illumination conditions from scratch. These innovations have a limited impact individually, but added up, they hold the potential to make augmented reality a much smoother and realistic experience than it is today.