“The Holodeck is Other People” was one of the first things Ken Perlin told me when I joined this lab, well before it was the Future Reality Lab. If you’ve listened to him or read his blog, you’ve probably heard it as well. It’s a statement that has driven many of the projects at our lab, and as I work towards writing my thesis, it has been at the front of my mind at all times.
When I look at most Virtual Reality applications these days, they are built on the foundation of providing a good user experience for a single person. Even when I think about multiplayer games I have seen on Steam, other people are a secondary consideration. Generally, they are functional as single-player experiences, where other people just add to the experience instead of defining it. Under this definition, does this describe everything? Is there anything that is inherently built for multiple people?
All of our FRL members contribute to this blog biweekly. When we do so, we do not write to please ourselves, we write these thoughts to share with other people, the vast population of the Internet. This blog, nor the vast majority of content on the Internet, would not exist if it were only to be seen by the people who created them. Even more primal than the Internet, humans have been sharing their ideas, their art, and their creations since… well, forever (see CAVE at SIGGRAPH 2018 if you need a reminder)!
So to me, that is the crux of social VR: I want to be able to create a virtual space that I can then easily share with others. If I have an idea, I want VR to enhance my ability to explain this idea. And through this shared experience, I form a stronger connection with the people I have shared it with.
Of course, there are already good content creation tools for VR. Tiltbrush, Tabletop Simulator, and other similar applications allow users to generate and share content, much like what I am asking. The application side of VR is quite rich. However, to me, it still lacks the connectivity that I want to see. I can view someone else’s Tiltbrush creation, but I can’t yet view it with other people and talk about it. And while there’s an obvious solution available to network multiple people together, I believe we’d be better off with a larger paradigm shift.
When I have an idea, I can write it down on a piece of paper, take it over to Sebastian a few doors down, show him, and we can discuss it. In our conversation concerning this idea, all that matters is the two of us and that piece of paper which has the idea. Now, as I do with many things, I like to imagine how this could be powered up with VR. What if I could create a small, virtual idea, say a sketch in Tiltbrush or Chalktalk, walk over to Sebastian a few doors down, and share this idea with him? And, much like the earlier example, what if all we needed was ourselves and the idea?
This is the foundation for my thesis. Inside-out tracking is becoming more and more common. Many A/VR devices have forward-facing cameras, like the Lenovo Mirage Solo, Vive Pro, and every phone used as a VR device. There is a complex problem in trying to completely solve your global head pose from what your camera sees (which is a great problem to solve, and many are working towards this goal), but I believe there is a much simpler solution.
If we can see each other via simple tracking solutions, we know where we are relative to each other. Then, we can create a local, shared virtual environment. Within this environment, we can share ideas and create content together. In an ad-hoc manner, other people could physically and virtually join and leave as they please, adding to our collaboration.
This foundation is quite different from the foundations that other VR systems are built upon. It doesn’t just encourage multiplayer usage, it demands it. Other people become a necessary part of our virtual world. To me, this is where VR becomes truly exciting, once “the Holodeck is other people.”
Like any foundation, there is still much left to discuss. What do we build on top of this system? What are its shortcomings? I will write about these topics in the weeks to come, but if you have ideas, feel free to comment below!