Calibration, Registration and Volumetric Capture

Since we got a pair of Intel Realsense Depth Cameras, I have been working on creating a pipeline for volumetric capture at the lab. However, if we were to capture footage from different depth cameras we would need a solution for stitching up the output pointclouds, that is where my investigation began.

First, I created a way to decode the bag files exported from Realsense’s Depth Camera. These files contained all the information necessary to combine point clouds together, including RBG (texture) and the xyz coordinates from the values of the depth map. At first, I attempted to use ICP global registration to stitch multiple point clouds together, but very quickly found out how it would not always work if the point clouds were not aligned and if there were not significant (50%+) overlap between the two PCs. That’s when I turned to camera calibration.

Then, Zhenyi and I worked together on streamlining a pipeline for camera calibration. Once, that was used to get the stereo calibration, of two cameras, we had the matrix transformations to be able to bring one point cloud into the world space of another!

Two separate point clouds of my face.
Point clouds stitched together after Global Registration.

I was also able to merge the textures onto the point cloud as well.

Texture point clouds.
The two points clouds stitched together using, camera calibration and registration.

2 thoughts on “Calibration, Registration and Volumetric Capture”

  1. Richard McLennan

    Hi this is really great work. Are you far off from a shareable solution to the mutli realsense capture?

    1. Alexandra Serralta

      Hello! By multi-realsense capture do you mean the merging of point clouds from different realsense cameras?

Leave a Comment

Your email address will not be published.