Rather than the more common virtual reality platforms, HoloLens is an augmented reality system which locks floating images within the user's vision - dubbed, inaccurately, "holograms" by Microsoft - and allows full 3D interaction, albeit in an extremely narrow field of vision. This AI coprocessor will form part of the next custom multiprocessor that powers HoloLens called the Holographic Processing Unit (HPU). The AI coprocessor will analyze visual, auditory, and other sensor data locally on the device instead of relying on sending the information to a cloud-based server for review.
The second version of Microsoft's HoloLens, the firm's "mixed-reality" headset, will feature an on-board AI coprocessor designed by Microsoft itself, Microsoft's AI lead Harry Shum announced in a keynote address at CVPR 2017 on Sunday. But today Microsoft has given us some details of the HPU of their upcoming version of the headset Hololens 2.0.
Marc Pollefeys, director of Science at Microsoft, explains the reasons for this: "Although we have seen large improvements in the accuracy of recognition as a result of Deep Neural Networks (DNNs), deep learning approaches have two well-known challenges: they require large amounts of labelled data for training, and they require a type of compute that is not amenable to current general objective processor/memory architectures".
That requires Deep Neural Networks, which right now work best in the cloud running on custom processor/memory architectures.
Microsoft isn't the only one creating its own chips.
Pollefeys then states that he works "on HoloLens, and in HoloLens, we're in the business of making untethered mixed reality devices". 'The AI coprocessor is created to work in the next version of HoloLens, running continuously, off the HoloLens battery.
The second version of the HPU, now under development, will incorporate an AI coprocessor to natively and flexibly implement DNNs. This new AI chip will support a wide variety of layer types and HoloLens team can customise it to their needs.
Microsoft has many times reiterated that they consider Mixed reality and artificial intelligence as the future of computing.