Tensorflow bodypix

2 min readMay 21, 2021

Key code

const img = document.getElementById('image');
const net = await bodyPix.load(/** optional arguments, see below **/);
const segmentation = await net.segmentPerson(img);

Model Config

use ModelConfig to load different config

1. architecture

key: architecture
value: MobileNetV1 | ResNet50

MobileNet (smaller, faster, less accurate)
ResNet (larger, slower, more accurate) **new!**

2. multiplier

for MobileNetV1 only

key: multiplier
value : 1.0 | 0.75 | 0.50

The larger the value, the larger the size of the layers and more accurate, but lower speed.

0.5 is for mobile

3. quantBytes:

how many bytes per float.

key: quantBytes
value:4 | 2 | 1

The larger the value, the higher the accuracy and bigger model size.

4. modelUrl

optional.

An optional string that specifies custom url of the model. This is useful for local development or countries that don’t have access to the models hosted on GCP.

Key Feature

1.Person segmentation

const segmentation = await net.segmentPerson(image, {
  flipHorizontal: false,
  internalResolution: 'medium',
  segmentationThreshold: 0.7
});

2. Person body part segmentation

const segmentation = await net.segmentPersonParts(image, {
  flipHorizontal: false,
  internalResolution: 'medium',
  segmentationThreshold: 0.7
});

3. Multi-person segmentation

const segmentation = await net.segmentMultiPerson(image, {
  flipHorizontal: false,
  internalResolution: 'medium',
  segmentationThreshold: 0.7,
  maxDetections: 10,
  scoreThreshold: 0.2,
  nmsRadius: 20,
  minKeypointScore: 0.3,
  refineSteps: 10
});

4. Multi-person body part segmentation

const segmentation = await net.segmentMultiPersonParts(image, {
  flipHorizontal: false,
  internalResolution: 'medium',
  segmentationThreshold: 0.7,
  maxDetections: 10,
  scoreThreshold: 0.2,
  nmsRadius: 20,
  minKeypointScore: 0.3,
  refineSteps: 10
});

Output Visualization Utility Functions

bodyPix.toMask
bodyPix.toColoredPartMask
bodyPix.drawMask
bodyPix.drawPixelatedMask
bodyPix.drawBokehEffect
bodyPix.blurBodyPart