When I give mixed reality presentations in Chalktalk, it bothers me that the 3D computer graphic objects floating in front of me look fake. The problem is mainly that the lighting of the virtual object doesn’t match the lighting in the room I’m actually in. You can see an example of this here: To fix
Immersive audio is necessary for any convincing XR experience. However, many times it is not clear how to create a natural audio layer when trying to go beyond traditional headphones and stereo speaker systems. Today I would like to introduce an object-based approach to sound mixing which can simplify this process. Spatial audio allows positioning
Object-Based Audio Read More »
Today, I am happy to write that the paper relevant to one of my previous posts has been accepted! Specifically, it has been accepted as a Late Breaking Work to this year’s SIGCHI conference. Zhenyi, Professor Ken Perlin, and I collaborated on this paper. For the project, we implemented multiple ways of configuring content and
Since we got a pair of Intel Realsense Depth Cameras, I have been working on creating a pipeline for volumetric capture at the lab. However, if we were to capture footage from different depth cameras we would need a solution for stitching up the output pointclouds, that is where my investigation began. First, I created
Calibration, Registration and Volumetric Capture Read More »
I am getting closer and closer to graduation, which means that I am getting busier and busier. Sometimes, I find it helps to take a mental break from intense focus and think about something else for a little bit. So, today I’d like to write about something I find fun to program. One of the
Two weeks ago, I participated in the Global Game Jam at the NYU Game Center jam site for the first time. I was very fortunate to have the opportunity to work with some amazing artists and engineers, including two of our lovely FRL Lab members, Pasan and Marcus. It was a stressful but exhilarating experience
Housewarming (GGJ 2019) Read More »
This past May, Google announced an open source computer vision and deep learning project called PoseNet for TensorFlow.js that is able to estimate human poses using only Javascript. I decided to check out the library and see what type of results I could get using the webcam on my laptop. Open Pose, the previous real-time
Pose Estimation in Javascript with TensorFlow.js Read More »
During a recent night out, a small detail of restaurant decor really caught my attention and presented itself as a fun computer vision problem. Here, let me show you some pictures. Do you see what I see? The bar lighting! The shades are made from repurposed player piano rolls! I spoke with the barkeep who
Inspiration strikes a chord! Read More »
With the start of the new semester this last week, I realized that I have only 2 months to finish my museum. There’s a lot left to do in regards to the modeling and visual development of the structure but it occurred to me that I hadn’t put a lot of thought into the player
Movement in the VR Museum Read More »
Having just begun my work as the administrator last week, there’s already many new things that I’ve started working on. Recently I’ve been testing updates for the FRL website. With these updates readers (like you!) will become more familiar with the lab’s members, as well as the fantastic work they’ve done. In particular, I am
