Today I wanna discuss a classic topic: Calibration.
I was doing stereo camera calibration for making a sketchy AR Rift four years ago. I tried to calibrate the first version of Leap Motion with usb camera three years ago. I did a rough calibration between OptiTrack and HTC Vive one year ago. And I spent several months to figure out how to calibrate Hololens with OptiTrack.
Calibration everyday calibration everywhere.
I didn’t realize I have been dealing with it for this long.
The fun part is I had a conversation with a new friend here. He accidentally mentioned he did gesture detection with OptiTrack in an AR environment, aka Hololens. For now I still think HoloLens is THE optical see through AR glasses, if they can make the fov larger. That experience totally attracted me. I have been struggling with the precise calibration between Hololens and OptiTrack for a long time, which ended up with coarse calibration results.
What a coincidence, he almost did exactly the same thing as I did. 1) We attached markers to the hololens. 2) We place some known model with markers in front of the user and ask user to move the physical model to overlap. 3) We have same issue about converging with both eyes and we have to cover one eye to do the overlapping movement and switch later. Even in this setup, we cannot get precise results and it is not a smart calibration anyway. Friend told me he had around inch-ish offset and I felt I am not alone. The difference is friend was using controller to shift the model in case user cannot do the overlap task perfectly by hand, and I used other ray-cast approach to improve. (Keep it secret for now 😉 )
Meanwhile we extend the idea to record AR view because AR calibration is different from person to person. The best way must be some complex setup to hold the headset somewhere. Regardless Hololens has its own inside-out tracking, which probably conflicts with OptiTrack’s outside-in tracking.
For anything related to cameras, anything related to two coordinate systems, we need calibration. It is so basic and so difficult and so much fun.
See how we dress our model with a virtual cloth. Hope we can do a more precise job later!