NYU FRL co-located multi-participant extended reality

Our open source WebXR platform focuses on supporting video pass-through co-located XR experiences. While companies like Apple and Meta are focusing on single-person or remote multi-person communication, there has not been much emphasis on computationally enhanced creative collaboration between people who are physically in the same room and facing one another. Our platform brings computer assisted collaboration to the physical space between people, centered on our human bodies. This platform promises to empower human communication, just as the Internet, the Web and the Smartphone did when they were introduced.

A collection of demos showcase what we can do with this platform. Our platform offers the ability to quickly build multi-person co-located XR experiences for education, entertainment, and other purposes. Below are INSTRUCTIONS followed by a LIST OF DEMOS (WITH VIDEOS).

INSTRUCTIONS

To play our demo, go to:  https://nyufrl.com:2024/

How to synchronize the space for multiple participants

You can move around the entire virtual world by squeezing both controller grips and moving your two controllers.

If two or more participants take turns placing both controllers in a given location in the room and squeezing both controller grips, their views into the shared virtual world will be spatially synchronized with each other (in a room with known anchors, such as our lab, this process happens automatically).

For the chess game, the world of the black player is automatically turned by 180 degrees, so the two players can just face each other, touch their controllers together and squeeze their two grips.

How to adjust the height of what you are seeing

Squeeze the grip of one controller and move that controller up or down.

Note: If you put your controllers down, after a few seconds you will see virtual fingers superimposed on your hands. You can then use your hands just like controllers within any demo. Pinching with your thumb and forefinger is the same as squeezing a controller’s trigger. Pinching with your thumb and middle finger is the same as squeezing a controller’s grip.

DESCRIPTIONS OF DEMOS, WITH LINKS TO VIDEOS

Magic Crayon

An XR magic purple crayon to make drawings in the air that then come to life.

Statue

A shared XR object that appears to be a realistic part of the physical world

3D Bar Chart

A glimpse into what the future of shared XR infographics might look like.

Live coding of an animated scene in shared XR. Everyone sees the evolving animated scene as though it is in the room. Everyone sees the code window as though it is facing toward them.

Multi-participant XR construction activity. Rules: Click in the air to create an object. Drag an object to move it. Click on an object to delete it.

Two-person XR chess game. Switchable to either tabletop size or giant size. NOTE: The world of the black player is automatically turned by 180 degrees, so the two players can just face each other, touch their controllers together and squeeze both their controller grips.

Hypercube

An XR hypercube floats in the room, projected from 4D down to 3D. Drag to rotate x↔w, y↔w or z↔w.

Moon

A 10-foot diameter XR moon floats over all of our heads. If we want, we can pull the moon down from the sky to get a closer look.

Morphing Sculpture

An XR brass Steam-punk sculpture morphs between a dodecahedron and an icosahedron.

Magical Window

An XR magical window into another world. The other world can only be seen from one side of the window.

Puzzle

26 puzzle: A 3D XR analog to the classic 15 puzzle. It can be played by multiple players.

Skylight

An XR skylight appears in the middle of the ceiling that we can all see, as though we are looking up into the sky together.

Smoke

Somebody left an XR cigarette burning.

Snow

Falling XR snowflakes fill the room, composed of 50,000 textured particles.