Pose Estimation in Javascript with TensorFlow.js

February 4, 2019byMichael Gold

This past May, Google announced an open source computer vision and deep learning project called PoseNet for TensorFlow.js that is able to estimate human poses using only Javascript.

I decided to check out the library and see what type of results I could get using the webcam on my laptop.

Open Pose, the previous real-time pose estimation system I had used, required a powerful GPU to get results at 24 frames per second, and would only run at 0.5 frames per second on my laptop.

I was not expecting to get results in real-time with PoseNet. However, I was in for a surprise.

Running the demo site for PoseNet, I was able to detect poses for multiple people at 20 frames per second.

Real-time pose estimation at 20 frames a second on a laptop with an integrated graphics card

It even worked on my mobile phone’s browser

One of the nice thing here is that all of the code is executed in the browser and no data is sent to a server. Therefore, this runs locally without the need for expensive GPU hosting.

A new paper about TensorFlow.js submitted to Arxiv last month dives deeper into how TensorFlow.js works and how to convert existing models for use with this library.

My prediction for 2019 is that we should expect to see more web apps taking advantage of this platform.