A Glimpse of Our Holodeck project

Our lab is collaborating with a few labs of NYU on a big project called Holodeck. The basic idea of the project is to create a platform for players co-located all over the world to join and interact with each other in a virtual world.

The project Urbane from Visualization Imaging and Data Analysis Center attracted my attention. It is a 3D framework to support data-driven decision making in urban development. In the software, users can manipulate the position and shape of the buildings, design or change a neighborhood. And the amazing part is that the shadow of the buildings, the planning of the roads, and a lot of stuff are adaptive to the editings in real-time. We wanted to utilize it in the big Holodeck project for players to design a block or even a whole city responsively. The role of our lab is mostly taking care of the tracking part. If players are in a specific space and tracking needs to be super accurate, we are going to use Optitrack for the motion capture. But we are going to implement another setup which uses Rokoko motion capture suit to track players’ body motion when the tracking space is somewhat unlimited. For example, if we are simulating a city scene where people need to walk in a building with multiple floors, Optitrack and other leading precision motion capture systems can do nothing if players want to physically go up and down between different floors or want to move freely in a larger space than the few hundreds sqft the systems can track.