CS491 - Virtual and Augmented Reality

Microsoft Holoportation | A Critical Analysis

What is Holoportation?

Holoportation is Microsoft's version of realtime teleportation with 3D Holograms. Holoportation enabes you to see, hear and interact with remote objects in realtime.

How it Works?

The object to be holoported is placed in the predefined zone, which is surrounded by a number of cameras that capture the depth information of the subject. These cameras capture the subject in 3D, thereby making it temporally consistent. Once the object is captured, a holographic 3D model is constructed. Then, textures are applied on this 3D model. This model is then compressed and transported. This information is received by the Hololens or any other similar system which then decompresses it, extracts the model and renders it in the user's space. Everything happens in realtime so that, every action or movement done by the subject in its surrounding is reproduced in the viewer's environment.

Requirements

The major chunk of work in holoportation goes into capturing the object in 3D. The second major task is to building a 3D model based on the captured information and finally it is compressed. To capture the models, high quality cameras that are capable of capturing depth information of a subject are required. Ideally, these cameras are placed at multiple locations surrounding the subject, images can be captured from maximum number of angles. To receive and view these captured models, a head mounted display such as Hololens or HTC Vive is required. These two are the basic necessities of implementing Holoportation.

Collaboration

With Holoportation, remote collaboration can be achieved. Anyone who wants to collaborate can be placed in their base station where the model will be captured. This user will also be wearing a HMD so that he will be able to view the holopeorted version of his collaborator. In this case every user's station will be recording and transporting models while the user will be receiving from other collaborators.

Limitations

Cameras:

3D Cameras that capture the object are vital to working of Holoportation. As we saw that the cameras are placed around the subject to be holoported, it should be noted that the quality and accuracy of the captured model would be dependent on the number and accuracy of the cameras employed. When an object is captured from all possible angles spatially, it helps us to reconstruct the object in 3D with high accuracy. With lesser number of cameras, low quatily objects would be built and vice versa.

Compression Algorithms:

The captured 3D objects are compressed and then transported to the viewing system. This compression algorithm should be a lossless compression because, the quality and accuracy of the holoported image should be the same or almost same as the original object. If not, the whole purpose would be lost and it would not give the immersive effect expected. So, the compression and decompression algorithm should also be taken care of.

Bandwidth:

Even after compression, the realtime data of 3D objects tend to be very huge. To have the object and viewer in sync, this data should be transmitted immidealtely to the viewer without any lag. If not, it gives a very bad effect. So the bandwidth of the channel over which the data is to be transmitted should be high enough to handle the data at sufficient speed.

Current Updates

Microsoft Holoportation was initially implemented for fixed base stations. Now, MS has extended it to the mobile platform of Automobile. Microsoft claims it has reduced bandwidth requirement by 97%, but this has not been shown in demos.