Blog

In collocated VR classes, instructors need to guide their students, while also remaining aware of the physical environment in order to ensure students’ safety. It is hard to do both simultaneously. We present a system that utilizes hand-held devices for non-VR instructors, enabling them to explore VR content and interact with students who are fully … Asymmetrical VR for Education Read More »

In 1992 I first read Neal Stephenson’s novel Snow Crash. I was influenced by many things in that book, but one thing that struck me in particular was the way the Metaverse (a term Stephenson coined for that novel) relied on good facial expression. In the story, Juanita, an expert on facial expression, implements the … Expressive faces Read More »

When I was a kid, I read about a story written by an Italian poet. It was about a kid stepping into a country where everybody has to spoke lies only everyday. While reading this story, I was thinking, “How can I come up with a sentence that is always a lie?” After a few … Halting Problem and Computer Simulation of Real World Read More »

What Director/Actor AR will look like in 2033: January 2033, Mr. Ahmed, Chen, Mark, Arpan: Mr. Ahmed walks to the door of his Vlass 1003, Vuilding A12, Sector EM. A slight pause and a gaze, the door disintegrates into pixels “Welcome to your history class, Mr. Ahmed”. Getting into the class, he finds it to … Director – Actor: 2033 Education in VR Read More »

Over the last three years, through the course of the pandemic, we have made various changes to how we operate. We have moved from going to work or class everyday in person to a completely remote approach where we hardly meet in person to get something done. Zoom along with the associated suite of tools … Questions Read More »

This is the year of 2022 AD, which is the year of 3 after COVID, is the universe meta enough? The memory of COVID is still fresh, while the impact of COVID is fading out from our life. When we look back to the past 3 years, nothing has really changed, except remote meeting becomes … 2022 AD, 3 AC Read More »

As we emerge from the experiences from the past few years, I think that we have learned (or re-discovered) the value of human connection. There is a renewed interest in supporting those connections better. You can see that interest represented everywhere: definitely in the flood of research coming from academia and industry. Collectively, we are … Forward Read More »

I have been playing with the first audio sunglasses from Bose, Bose Frames. With two speakers integrated separately into the temples sitting in front of the ears, Bose Frames beam music right into your ears and provide open-air audio experience with nothing in or on the ears. One of the reasons I wanted to play … Journey Started on Spatial Audio Read More »

One of the goals of our lab is to allow people in a shared mixed reality to collaborate together in manipulation of objects. Those objects might be real solid physical objects, or they might virtual objects, which exist only in our collective visual and auditory perception. One challenge is to create a seamless way of … Grasping real and virtual objects Read More »

I have been working on a platform combined with HTC Vive and pressure-sensing technologies to assess human balance in the walking paradigm. With the two new Vive trackers, we can now track foot position and orientation. Vive base stations can track the Vive trackers, so the setup only requires two Vive trackers mounted on the … Walking Balance with Vive Trackers on the Ankles Read More »

In the next few years SmartPhones will be augmented, and eventually replaced, by wearables. When that happens, we will be able to see and hear digital information floating in the air between and around us, even in casual conversation. This will have profound effects on everything from work to education to entertainment to social interaction. … Augmented Reality and the Future of Language Read More »

One of my heroes in the field of computer graphics, Lance Williams, once told me that one of the wonderful things about CGI is that it “is limited only by your imagination.” Accepting that realization gives us enormous power as creators, yet Peter Parker’s uncle Joe was also right when he cautioned that “with great … Personal values Read More »

I recently took a look into the MultipeerConnectivity framework by Apple when I worked on an iPhone-based AR project. The MultipeerConnectivity framework is a lot easier for Apple’s ARkit to create a multiuser AR experience. Unlike conventional connections, the MultipeerConnectivity does not have to assign hosts or guests if you would like to define all … A Glimpse of MultipeerConnectivity Read More »

I am currently working on ways to interact with four dimensional objects in virtual and mixed reality. It’s pretty much the realization of a childhood dream of mine. I always wondered what it would be like to be able to interact with objects that have four spatial dimensional, right here in our own world. As … 4D Read More »

When the countdown run from 10 to 0 and the ball dropped in Time Square, we had to wave goodbye to 2019 and to all the happiness and sadness that came along with it. In the meantime, our new journey irreversibly boarded on the ship of 2020 which has just been christened by its champagne … Happy New Year Read More »

When I was a kid I would watch a cartoon called Tennessee Tuxedo and his Tales on Saturday morning TV. Tennessee Tuxedo, who was a penguin, would go on all sorts of adventures with his pal Chumley, who was a walrus. Whenever they ran into a problem they couldn’t solve, they would go to their … 3dbb Read More »

Some people wanted to Disable Guardian on the Oculus Quest (thanks to Mike for sharing the helpful instruction). A lot of them might have the requirement for obtaining the guardian boundary in their apps. In this blog, I am going to share the method to obtain and visualize the guardian boundaries in your Unity app. … Obtain Guardian Boundaries on Quest Read More »

This evening the students in my graduate computer graphics class showed their final projects. It was quite unlike anything I have ever seen. The projects were group experiences, with every participant wearing an Oculus Quest VR headset. With the help of WebVR software our research team had provided, the students created shared amazing interactive shared … Final projects Read More »

One of the most frequent complaints I hear about developing for the Quest is: I have to confirm the guardian every time I put on my headset – and this can be many times during a coding session. If you’ve always wanted to disable these notifications, you’re in luck. With the latest firmware installed on … Disable Guardian on the Oculus Quest Read More »

I have been playing with Pytorch recently and found out that Pytorch is a much easier tool for me to start a machine learning project than Tensorflow. Pytorch uses a different design with Tensorflow and does not need to setup sessions or placeholders, so I feel the setup of Pytorch models is more intuitive. Pytorch’s … Pytorch Foot Image Classification Read More »

Today at the start of my graduate graphics class we showed a VR metaroom demo with multiple people in the classroom wearing Quest VR headsets, holding controllers and collaborating over WebVR. We made it even more interesting by having the entire scene — the room with all of its furniture and the avatars of all … Seeing the future take shape Read More »

For a project during my masters degree, we implemented the paper Learning to Generate Chairs by Dosovitskiy et. al. in PyTorch. The only implementation available was the author’s in Caffe with Lua, so we set off to create a clean open source implementation of the work. This work presents an early differentiable renderer using convolutional … Learning to Generate Chairs with Convolutional Neural Networks Read More »

1.This week I found a paper, SDM-NET: Deep Generative Network for Structured Deformable Mesh. Their main idea is to decompose a 3D object into parts. And each part can be deformed from a homeomorphic simple object. Their network can also learn the support and symmetry information from the input. The part structures and geometries are … Paper reading: SDM-NET & New tools for 3D deep learning Read More »

This week I setup a 15-meter walkway in the lab for the obstacle experiment in walking paradigm. I chose to use Vive pro with a wireless adapter to accomplish a wireless VR experience. Compared with Oculus Quest, the wireless Vive pro is much heavier, and it is still constrained by the coverage of Lighthouses and … Tested a 15m Walkway Read More »

I started an experiment if it’s possible to take a screenshot of a camera that takes a screenshot of what the camera is seeing which then is rendered to a photograph in virtual reality. Although I expected the performance wouldn’t be too well since it’s using the Oculus Quest, but it turned out it worked … Virtual Reality Wildlife Photography Simulator Read More »

This last week or so I’ve been building a parallel world. I measured all of the sizes in a section of our lab — walls, floors, tables, and so forth. Then I built a computer graphic copy of that world, matching the real thing down to the centimeter. I can walk around in my parallel … A parallel world Read More »

Over the years, I’ve work on a side project – a C++ OpenGL rendering engine that I use as a staging ground for experiments with procedural generation. One feature I had particularly had fun implementing, was adding heightmap rendering, with a node based graph built ontop of DearImGUI, with a mask painter alongside (the effects … Node Based Terrain Generation Read More »

Last week I finish the function for the pose skeleton normalization. Given a referenced skeleton expressed with global coordinates, we can take some joints of it(instead of all joints) and normalize the length of the chosen joints of the skeletons in other frames. All the computations are using PyTorch’s tensor, which can be easily used … Normalize the pose and visualization Read More »

A few weeks earlier, I talked about two papers I read regarding gait balance when fixed or suddenly appeared obstacles are involved. The papers are Age Effects on Strategies Used to Avoid Obstacles, and Anxiety-mediated gait adaptations reduce errors of obstacle negotiation among younger and older adults: Implications for fall risk. We have been looking for protocols and … Virtual Obstacles in Gait Read More »

Last week, I’ve finally successfully installed Unity ML Agents after several attempts in the past. As a person who has a design & research background, I find it very interesting to test tools that implement machine learning. I hope to learn more about the basics of machine learning which I believe could be complimentary to … Installing Unity ML Agents Read More »

Today I surprised the 27 students in my graduate computer graphics class. I handed out an Oculus Quest VR system to every student. They will each get to use the Quest for the rest of the semester. They will be able to take the Quests home with them, and use them any way they like. … Christmas in November Read More »

I’ve been thinking a lot lately about networking and how we use it for our co-located immersive experiences. What are the pressing questions as one designs such a system for shared experiences like these? In order to deliver these compelling experiences that exceed the capabilities of stand-alone, inside-out tracked devices such as the recent Oculus … Distributed Systems and Virtual Reality Read More »

Last week, I had to take a week off to attend the final demonstration event for Niantic’s Beyond Reality Developer Contest. It’s a 4-month hackathon where participants create an augmented reality game that encourages exploration, exercise, and real-world social interaction by using Niantic’s Real World Platform SDK. Niantic’s Real World Platform includes a world-scale persistent … Niantic’s Beyond Reality Developer Contest Read More »

Last week I helped Zhenyi install the Ubuntu 18.04 dual system, and install NVIDIA driver, CUDA-10.0, CUDNN, Pytorch-gpu. Although there are many tutorials on the Internet, only very few works. So I think it is better to make a record. BIOS setting Shutdown and reboot. And enter the BIOS interface. Disable the Secure Boot. If … Tutorials for installing Ubuntu 18.04 dual system and NVIDIA DRIVER-CUDA-Pytorch GPU toolchain. Read More »

On Oct 25, we are going to present a poster in a research showcase organized by The Center of Health and Rehabilitation Research (CoHRR). The CoHRR provides opportunities for networking and discussion in all kinds of research fields related to physical rehabilitation. The project we are going to present is our foot analysis system. The … Healthcare Showcase @CoHRR Read More »

My favorite thing to do when I teach my graduate computer graphics is live coding. Don’t get me wrong. I love working through mathematical principles on the whiteboard. I really enjoy free-ranging conversations with the class on various topics. But live coding is in its own category. That’s the moment when the class sees me … Live coding in the classroom Read More »

UIST 2019 is coming close. I went through the accepted papers this year quickly and preferred to discuss some interesting ones. First of all, I did not see many interesting papers in my area, which is good news because that means my literature review work could be large decreased this time. I split the papers … A Quick Glance at UIST 2019 (Part I) Read More »

Social presence, intimacy, & user agency is still an occurring issue in virtual reality experiences that involve social interactions. While remote virtual reality social media has been reiterated numerous times (i.e. AltspaceVR, VRChat, Rec Room), I think a physical shared space virtual reality social platform would be interesting to see in the future, especially with … Virtual Reality as a Physical Shared Space Social Platform Read More »

Kristofer and I found a paper Learning Implicit Fields for Generative Shape Modeling [https://arxiv.org/pdf/1812.02822.pdf], which is about using the implicit function to do 3D reconstruction. They use a DenseNet-like structure. Unlike DeepSDF, they use a binary output. Given a latent vector and a 3D coordinate, the network will output 0/1 which expresses outside/inside. Its structure … Implement a paper of 3D reconstruction. Read More »

Our balance assessment research is currently at the point of transforming from a stationary standing paradigm into the dynamic walking paradigm. In the standing paradigm, we explored the assessment of anxiety-caused imbalance through various intensity levels of sound and visual flow. However, the dynamic and stationary paradigms are so different that we cannot use the … Virtual Obstacles in Gait Balance Assessment Read More »

A few years from now you go out to a local hangout with your date. The two of you, together with lots of other people in the room, put on your XR glasses,. Suddenly you are all sharing another world. You can walk around in that world, listen to stories, play games, experience moments of … Your future bartender Read More »

A long-time friend and collaborator at the lab, David Lobser (known for working on projects such as Flock) organized weekly VR-meetups at our lab. (Credit goes to David for the images in this post)These featured talks and demos by local VR artists and engineers, and offered us a chance to connect with the community. The … VR Meetups: The Phoenix will Rise Again Read More »

Recently we are working on a walking simulation. The task is to extract the valid part from an entire walking path. To be clear, an entire walking path is a sequence of pressure images provided by a pressure sensor. Each pressure image is a large image, of which only a small proportion is the valid … Extract Feet Bounding Box from an Entire Walking Path Read More »

The recent explosion of deep learning has demonstrated its awesome power, but data driven machine learning isn’t the end of the story. Machine learning is only one piece of the puzzle, in the future it will just be, as Ken has put it, “like a sine function”, could find use everywhere. It will be used … Convergence Read More »

Description As a combination of Computer Graphics and Computer Vision, 3D reconstruction has been a classic problem for a long time. In recent years, with the development of Deep Learning, more and more researchers are focusing on 3D reconstruction with Deep Learning again. Related Work The number of work focus on 3D reconstruction is increasing … A brief overview of single-view 3D reconstruction based on Deep Learning Read More »

There was a previous blog I wrote about Tobii eye tracking system embedded with HTC Vive. A few friends who are interested in it asked me about my experience with the eye tracking system, so I feel it is good to talk a bit more about it in this blog. There are currently two SDKs … A Preliminary Test with HTC Vive Pro Eye (2) Read More »

This is my first post here as a new lab member of Future Reality Lab! I’m Adlan Ramly, a graduate student majoring in Integrated Digital Media specializing in Human Centered Design & Research for Emerging Technologies. Previously I interned at The Smithsonian Institution as a Creative Technologist where I worked on ExplorAR: A Collaborative AR … Tangible Building Blocks in VR Read More »

I was asked to give a lightning talk this week, part of a series of lightning talks on related topics. I was told that the talk is to last 15 minutes, and that I can’t use more than five slides. I found this contraint to be highly refreshing. Rather than meander all over the place, … Five slides Read More »

I have been rather concerned recently that major industry players are missing the boat on the potential of virtual reality. This was confirmed for me this past Friday, when I attended an industry panel on the future of VR narrative. To my bewilderment, the panelists described market scenarios and economic models that were, on the … VR is other people Read More »

In preparation for the fall graphics class, I have been working on a basic “hand-made” WebGL live shader editor that accepts input from external text editors. The editor interacts with a simple server that checks for file and directory updates. The following video shows me slowly editing the shader on the left using Vim on … Hot Reloading Part 2 Read More »

Last week, I flew out to Cleveland, Ohio with some others from the NYU CREATE Lab to demo our educational, biological cell builder. On Tuesday of last week, we allowed both middle school students and teachers to try out our science lessons for themselves. For this demo, we were able to make the front end … EdTech Demo @ Middle Schools Read More »

Recently, I had a chance to test the new released HTC Vive Pro Eye headset. The headset is the first VR headset integrated eye tracking, and it is hard to notice the hardware for eye tracking when compared with a regular HTC Vive Pro. Vive Pro integrated the eye-tracking technology from Tobii, which is a … A Preliminary Test with HTC Vive Pro Eye Read More »

In the NYU Future Reality Lab we spend much of our time thinking about the near future, because that’s what we can actually prototype and then study. But what about the far future — not five years from now but perhaps fifty years or more from now? One thing that I’ve been thinking about recently … Future vision Read More »

It’s been a while since SIGGRAPH 2019, but I’d like to share a few of the highlights: Interactive Hand Pose Estimation using a Stretch-Sensing Soft Glovehttps://igl.ethz.ch/projects/stretch-glove/ Using a new fabricated material of the authors’ design, this stretchy skin-tight glove can track hand poses without being tethered to a machine. The glove requires no external cameras … A Few SIGGRAPH 2019 Highlights Read More »

A small team here at FRL have spent these last few months experimenting with Oculus Quest and the Web* family of APIs, building tools to bootstrap immersive content creation for education (specifically, computer graphics education). For those unfamiliar with the APIs, here’s a quick introduction: First off, meet the Immersive Web, a working group with … Oculus Quest, WebVR, and WebXR Read More »

I went to Siggraph 2019 a few weeks ago and had some thoughts about VR. With the release of the various commercial VR systems from the technology giants, VR is no longer a mystery these days. More and more people in all kinds of fields began to see it as a profound tool rather than … VR for Patients with Chronic Pains Read More »

I wish it is still not too late to talk about the conference. I was planning to talk about is two weeks ago but… Anyway, there are not a lot of interesting papers this year to me. Instead, I caught up with quite a few old friends who only show up at the conferences. That … SIGGRAPH Experience Read More »

We are well into the era when what we expect to see computer graphics characters in a live action film. Unlike characters in animated films, CGI characters in live action films are expected to be at least plausibly fit into the real world — although we are usually talking about a fantasy version of the … The limits of digital production Read More »

This morning Kris Layng and I got up in front of several thousand people and accepted the award for Best Art Paper at the Siggraph conference. The paper was about CAVE, our shared VR experience for large audiences, first shown at Siggraph 2018 (and thereafter at the Tribeca Film Festival). Our coauthors on this are … Best Art Paper at Siggraph Read More »

Several of us are on the way to the west coast to attend this year’s SIGGRAPH. This will be my first, and I am looking forward to meeting people there. I’ll have an update when I return.

This post is to share the solution of a problem that happened during the development with Oculus Quest. I got the solution from Connor (he was a colleague who has a lot of experience on Quest and just graduated in the summer). I have been working on a Unity project which requires to record position … Write Data locally on Oculus Quest Read More »

This is a follow-up work after I listened to the talk of RealityCheck @CHI2019. RealityCheck explored how to composite the virtual environment and the real world without pre-captured geometry of the physical world. Combining both the virtual environment and the real world raises a sense of mixed Reality. That is very attractive to me and … Literature Review: Blending the Real World with Virtual Environment Read More »

I was going through some of my older experiments and I came upon this picture. It’s not much — just the visual interface for a simple computer graphic xylophone. Yet as I looked at it, I remembered how much time and care I had taken in designing the colors. My general plan was to go … Choosing colors Read More »

Header Photo: viewing New York City from the edge of Roosevelt Island Earlier this week, a few of us at the FRL visited Cornell Tech to meet with the Mixed Reality (MR) group. We were kindly invited to discuss our work and see whether we might collaborate in the future based on shared interests. (I … FRL Collaboration Read More »

Hi everyone, I’m sad to say that this will be my last blog post for the FRL, as I will be leaving here Friday after 5 years working with the lab. It’s been a great journey working with everyone here! My original plan was to write a retrospective on what I’ve done in the lab, … Moving Forward Read More »

At this point, I have finished cleaning the mocap data and the facial animations for all of the characters, except for the 3rd scene for the monster. I have moved on to the “contact pass” for all of the animation. For this pass, I’m reviewing all of the animations for each character and making sure … Animation Contact Pass Read More »

I am Keanan Pucci, a rising senior Tisch Game Design major who just so happened to be a PA for Parallux one day, and decided to stick around. As a new face at the Future Reality Lab, I’ve spent the past month exploring the different facets of this space and settling into my own work. … A Warm Welcome Read More »

Earlier this year, our lab got a lightweight IMU-based wearable motion capture system called Notch. The system is a set of six waterproof IMU sensors designed to be mounted on the body. The motion capture system does not use external cameras like Optitrack or Vicon which uses a group of infrared cameras to triangulate the … New Toy: Notch Read More »

Our Future Reality Lab has been focusing our efforts on creating extended reality experiences, by inviting people to share an alternate reality that exists nowhere in the physical world. But not all XR experiences are virtual. This past week I visited several amazing places in Silicon Valley. One of them was Facebook, which makes the … Another kind of extended reality Read More »

Here is a video of the current version of Chalktalk VR. Will (a summer intern) and I draw a simple scene together in the virtual version of our lab. I drew the boat in purple.

I’ve been gone from the lab for the last 2 weeks, visiting family in Sri Lanka. In that time, the Monster’s character rig had been finished. So, I spent this week completing the facial animation for the monster in scene 4. The facial structure of the monster was a lot different from the other characters … Monster Facial Animation Read More »

Hey Everyone! I thought it would be good to give a quick update on things to look out for on the FRL website. Here at the FRL we have many generous sponsors that enable us to do amazing research. In order to express our gratitude to them, I’m in the process of creating a sponsors … Website Update Read More »

Today’s post envisions the state of XR five years in the future, and how we might prototype advanced features like room-scale digitization with current hardware. This latest generation of HMDs (Oculus Quest, Oculus Rift S, Valve Index, etc.) has swapped out external tracking equipment in favor of HMD-mounted camera arrays. This change is more than … Stepping into worlds with a Reality Zipper Read More »

Rokoko motion capture suit is an IMU-based motion capture system with IMU sensors worn on the human body and recovers poses from measuring orientation and acceleration of each IMU sensor. The suit tracks all the major body components except fingers with 19 IMUs. Last week, Ben, Connor and I made a demo video showing how … Rokoko + Quest = Ready Player Zero? Read More »

Early this semester, we were working on volumetric capture. The project is about to pre-record human’s presentation and replay it remotely offline. The project includes: 1) Camera rig. To capture the whole body of the presenter, four walls of the cameras are the initial setup. 2) Movable frames. To support the movement of the presenter, … Volumetric Capture Literature Review Read More »

This morning I was working on an algorithm for being able to write text quickly by drawing with a VR/XR controller without needing to look at the controller. Technically this is called “eyes free” writing, because when you don’t need to look at what you are doing with the controller, your eyes are then freed … Accidental mandala Read More »

Recently, I think a lot about how to use the new technology to allow the users to immerse themselves inside the music. We are so used to listening to music in “flat” stereo that the opportunities of 6DoF tracking are very tempting to be explored in the music context. In one of my previous posts, … Immersive music Read More »

Most of us here are preparing a large palette of videos and experiences to present to visitors and potential partners. Within the general sphere of this activity, I am working to make Chalktalk’s multiple user support more robust, which requires a major rewrite of the core code-base. Since I’ve familiarized myself with the project fairly … Working on Multi-user Chalktalk Read More »

June 6th we took to Verizon’s Alley office to present our work for the past couple of months. A team of us at the lab have been working on a multi-user collaborative classroom. Specifically, we focused on creating a classroom lesson where students get together to build a cell from parts that are floating between … Verizon 5G EdTech Challenge Read More »

A couple of weeks ago, I wrote about a quick and easy calibration method. This calibration method allowed us to not only map a virtual space to a physical space with reasonable accuracy, but synchronize a virtual space between multiple users. However, as I used this method in practice with the new Oculus Quest, I … Calibrating the Oculus Quest Guardian Read More »

Now that we have room-scale co-located tracking with no external sensors, thanks to the Quest, and Connor DeFanti’s calibration scheme – we can begin to explore other exciting possibilities for the future. One of which is using small embedded boards, such as the Raspberry Pi, coupled with an camera – both attached to the front … Living at the Edge Read More »

The last few weeks I’ve been finishing the facial mocap for all the characters in the Mary and the Monster project. The work within the Faceware software is almost done, I only have 1 scene left for each of the characters. The work in the Faceware system has been slow and tedious. Some of the … Facial Animation Progress Read More »

I am sure most of you have seen Ken’s Chalktalk demos. In one of his demos, he uses a green object attached to a pen as the drawing tool. It would be nice to use bare hands as the tool for interaction, so we implemented this simple hand tracking component for Chalktalk. It uses the … Fingertracker for Chalktalk & Filter Blogs by Author Read More »

The problem with talking about the future is that it hasn’t happened yet. Sure, you can extrapolate from current trends, make predictions, follow patterns to their logical (or illogical) conclusions. But you don’t actually have that crystal ball, all polished up and ready for the gazing. So you’re sort of winging it. And the people … Giving demos from the future Read More »

This Sunday we will be presenting Chalktalk VR as part of the World Science Festival. See the festival’s site here: https://www.worldsciencefestival.com/ and our previous publication here: https://mattressiter.com/publication_post/exploring-configuration-of-mixed-reality-spaces-for-communication/The location is Kimmel Center if you’d like to try or take a look. This will be the first time the system as it is now will be run … World Science Festival Read More »

From time to time, I meander outside my typical work, and pursue a side project, often dealing with some random use of procedural generation techniques. For this one, over the past winter break, I relaxed by seeing if I could generate mushrooms with a few functions, and arrange them in interesting “pseudo” natural looking ways, … Procedural Fungi Generation Read More »

After participating 2 rounds of application to the second Annual Innovators in Aging Award, our team (Zhu Wang, Marta Gospodarek and Rufei Sheng, advised by Prof. Ken Perlin and Prof. Anat Lubetzky) finally won the award. The competition, held by NYU Aging Incubator, aims to promote the study of aging and its impacts, and further … We Won the Aging Incubator Competition Read More »

In the course of implementing a new computer graphics algorithm, I recently found myself immersed in the wonderful world of polyhedra. In particular, I got to play with two of my favorite polyhedra — the cube and the rhombic dodecahedron. The first one everybody knows about. The second one is a little more obscure, but … Shape shifting Read More »

Well everyone, this is going to be my last blog post of the semester, so I’ll try to keep things short. But, need not worry, I’ll be back again in the Fall, ready to write as much as I can (or as much as people will listen)! To be completely frank, these last few weeks … Wrapping Up the Semester Read More »

// Sorry for the late blog. I will share my personal feeling of CHI 2019 in brief here and attach a link to my full notes in the end. This is my second CHI experience. Differently, I have something to present this year although it is just a late-breaking work. Check the official preview video … CHI 2019 Read More »

Pressure sensing technologies have the advantage of obtaining the center of pressure. The center of pressure (COP) is defined by measuring the mean position of pressure distribution on the planar surface of the pressure mat. The COP of both feet are not going to be the same when we stand on the pressure mat and … Control Cursor with Pressure Read More »

There are so many papers to attend here at SIGCHI 2019 that the experience is quite overwhelming. At any given moment there are about 20 different things you can choose to attend, between all the courses, papers and other gatherings. There is a joke that going to SIGCHI is almost like not going to SIGCHI. … Gazing at SIGCHI Read More »

Zhenyi, Ken, and I are waiting at the airport to depart to Scotland. There, we will attend SIGCHI to present our Late Breaking Work Exploring Configuration of Mixed Reality Spaces for Communication. I’m looking forward to all of the events and meeting other people in the field. First comes the voyage.

Last Sunday, I successfully defended my dissertation on Co-Located Augmented and Virtual Reality Systems, thus completing my Ph.D. program here are the lab. It’s been a wonderful time over these past 5 years, and the FRL has been a great place to study. *Of course, this isn’t the end for me, but really just the … The End* Read More »

So, in my last post over I went over pretty much the main mechanic of my VR film: drawing. It took me nearly an entire semester to get it to a comfortable enough point where I could feel confident in moving on, but alas, here we are. Now comes the part that’s the most fun … Playing God for a Day Read More »

Balance control is a complicated process, which involves multiple systems such as the vestibular system, motor system. A lot of factors can cause imbalance but the underlying connections between the factors and imbalance are not well researched. We approach to reveal the primary factors and their connections to balance by analyzing the parameters that we … Overlook the next step of balance assessment Read More »

Recently Alex and I were working on Aruco board detection. In brief, our result was very unstable and seems wrong. We don’t understand why, especially the code we were using basically the official plugin code from OpencvUnity. So we split the team. She started to write the part for sensor fusion and I decided to … Aruco Detection with Built-in Cameras Read More »

Today I showed my undergraduate computer graphics class how to do bump mapping. That’s when you use an image to create a texture the gives the illusion that a surface is bumpy. It’s a technique originally developed by Jim Blinn in 1977. Instead of putting actual bumps on the surface (which is computationally expensive), you … We love computer graphics! Read More »

One of the things our lab has been doing for several years now has been taking people in the same physical space and placing them in a shared virtual space. One of the tricky things about this is that most virtual reality applications have some sort of built-in tracking system, but do not yet have … A Quick and Easy Calibration Method Read More »

So, despite the fact that I’ve now been developing in VR for one summer and nearly an entire semester, I still find myself going back to maybe the most simplest of things to do (in real life or in VR): picking objects up. Now, for seasoned VR veterans that might be reading this, I’m sure … Picking Up Isn’t So Easy Read More »