Virtual 3D Teleportation in Real Time with NVIDIA GPUs

Imagine being able to virtually teleport from one space to another in real time.
With 3D-capture cameras and a mixed reality display such as HoloLens, Microsoft Research’s new ‘haloportation’ innovation allows users to see, hear, and interact with remote participants in 3D as if they are actually present in the same physical space.
Custom software helps reconstruct every possibly viewpoint from each camera and then stitches them together into one fully formed 3D model.
“We want to do all of this processing in a tiny window — around 33 milliseconds — to process all the data coming from all of the cameras at once, basically, and also create a temporal model, and then stream the data,” says project lead Shahram Izadi, whose team leans on NVIDIA GPUs to crunch the relevant numbers.

Izadi also mentioned the system is able to record and playback entire previous ‘haloportation’ sessions which is like walking back through time to experience memorable events.
Read more >>