Extract Feet Bounding Box from an Entire Walking Path

Recently we are working on a walking simulation. The task is to extract the valid part from an entire walking path. To be clear, an entire walking path is a sequence of pressure images provided by a pressure sensor. Each pressure image is a large image, of which only a small proportion is the valid feet pressure. To extract the information we need, we need to:

1. Apply basic filter to get rid of the noise and to fill the holes. Based on the pressure image we already had, we found there were some noise points showing the pressure when there was nothing on that spot. We need to remove those points. We also found there were some holes inside of the pressure image. We decide to fill the holes for now.
2. Fix the bounding box for each step. We need to have a bounding box for each step, and the bounding box should be with a fixed size and shape. A changing bounding box is not useful for further process. So we need to
   1. Extract a rotated rectangle of the minimum area enclosing the pressure image. Originally I was using regular boundingRect, obviously, it only worked fine when the foot pressure is almost upright. Then I switched to minAreaRect, which returns a rotated rectangle. That is ideal.
   2. Accumulate the rotated rectangle for the same step. Apparently, the foot size of the same step won’t be different. Instead of extracting a changing bounding box, we should have one bounding box with the same property. So I calculate the center of the bounding box and determine if the step of the next frame is still the same step by the distance of the centers. The threshold is determined by the real data. It is adjustable and empirical. Once we know the current step is the same step for previous frames, we need to merge the current step to the previous bounding box and update the property. It is quite straightforward if we are using a regular upright bounding box. However, things become complicated for the rotated bounding box. Here I gave up calculating a rotated bounding box for two rotated bounding boxes. Instead, I recorded all the contour information. So I calculated the rotated bounding box for the convex hull of two contours. It worked quite decent.
   3. Filter the rotated bounding box by the time. Right now we will walk on the mat back and forth. To avoid the situation that we accidentally use the old bounding box for the new step. We used a timestamp to control that.
3. Fix the bounding box for all the steps. Currently, there is only one experimenter walking in each sequence. It is more reasonable that we have only one bounding box for all the steps. After step 2, we have quite a few bounding boxes. Now we calculated the largest bounding box of all and update the size property to all boxes.

See the gif for the current result.