BodyPix is a new segmentation technique that utilizes TensorFlow.js, a machine learning model released by Google. In addition to distinguishing people and backgrounds, it is also possible to recognize the human body by dividing it into 24 parts.
When a person is recognized using the body fix developed by the New York University Interactive Telecommunications Program (ITP) research team, the human body can be classified into 24 parts, including the head, torso, and limbs as well as the left and right limbs. The subdivision of Body Fix is largely divided into two stages. In the first step, the division of the person and the background, the body fix algorithm analyzes the image pixel by pixel, and evaluates the probability of being part of the person by pixel with a score from 0 to 1. The threshold of points that should be regarded as a person is determined, and a person is selected by setting the value below that value as 0, otherwise it is set as 1.
Next is the body distinction. Determine which part belongs to each pixel. When Body Fix was first introduced in February, only one user could handle it, but in the November update, it was said that it was possible to recognize in real time with multiple people.
One of the biggest features of BodyFix is that it does not require any special equipment and works in real time with a webcam attached to a laptop or a smartphone camera. BodyFix itself is also open on GitHub, and there is nothing that is being processed by a server somewhere through the cloud, so it works with only a PC or smartphone.
The research team has previously developed PoseNet, which can recognize a person’s posture in real time. The research team revealed that it is possible to easily capture motion in the outdoors, not in the studio, using a general PC and smartphone through Body Fix and Posenet. Related information can be found here .