Estimating the position of human limbs and joints recently proved useful on an image segmentation problem I've been working on. Fortunately, there exists pretrained models for human pose estimation in a natural scene. The paper Articulated Pose Estimation using Flexible Mixtures of Parts comes with the training and testing dataset along with full-body and upper-body pretrained models. The technique is particularly useful when we need to localize positions on the body, like head, torso, and legs. Pose estimation has a lot of applications, from object detection and tracking of human limbs and joints to my image segmentation problem of parsing clothing in natural scenes.

*Figure.* Example output of human pose estimation algorithm

I packaged their trained model along with a MATLAB and Python wrapper around the pose estimator to easily explore pose estimation in other settings (Github link). The package contains precompiled Mac OS X 64-bit binaries so it should work out of the box on a modern Mac.

Development install

git clone https://github.com/stitchfix/PoseEstimation.git
cd PoseEstimation
python setup.py develop  # Use sudo if necessary

MATLAB usage

Simply start MATLAB from the PoseEstimation folder to load the startup file. Alternatively, just call startup(). Run demo in MATLAB for an example:

You can also make a new pose object manually and visualize it.

im = imread(fname);
pose = poseest.Pose(im);
pose_img = pose.show('image',im);

pose.show('image',im) displays the joints overlayed on the original image and returns the image matrix. pose.show() just displays the connected joints as an image.

The poseest.Pose object has the following properties:

pose =

  Pose with properties:

               EDGES: [1x1 struct]
               LIMBS: [1x1 struct]
              head_x: 131
              head_y: 32
              neck_x: 131
              neck_y: 81
    right_shoulder_x: 81
    right_shoulder_y: 98
       right_elbow_x: 81
       right_elbow_y: 147
        right_hand_x: 81
        right_hand_y: 213
     left_shoulder_x: 163
     left_shoulder_y: 98
        left_elbow_x: 180
        left_elbow_y: 180
         left_hand_x: 213
         left_hand_y: 246
         right_hip_x: 98
         right_hip_y: 246
          left_hip_x: 147
          left_hip_y: 246
        right_knee_x: 114
        right_knee_y: 344
         left_knee_x: 147
         left_knee_y: 344
       right_ankle_x: 131
       right_ankle_y: 459
        left_ankle_x: 147
        left_ankle_y: 426
               score: 0

This contains the (x,y) coordinates of each feature point.

Python usage

Examples of calling the pose estimator from Python can be found in the IPython Notebook.

import os
from PoseEstimation import PoseEstimation

root_path = os.path.abspath('./') # path to PoseEstimation/lib folder
imgpath = os.path.join(root_path, "data/images") # path to folder of images
image_paths = [os.path.join(imgpath,fn) for fn in next(os.walk(imgpath))[2] 
    if 'jpg' in fn or 'png' in fn]
pe = PoseEstimation.PoseEstimation(image_paths, root_path)

labels,locations = pe.batch_estimate() # Start MATLAB once and compute poses on all images