Estimating the position of human limbs and joints recently proved useful on an image segmentation problem I've been working on. Fortunately, there exists pretrained models for human pose estimation in a natural scene. The paper Articulated Pose Estimation using Flexible Mixtures of Parts comes with the training and testing dataset along with full-body and upper-body pretrained models. The technique is particularly useful when we need to localize positions on the body, like head, torso, and legs. Pose estimation has a lot of applications, from object detection and tracking of human limbs and joints to my image segmentation problem of parsing clothing in natural scenes.
I packaged their trained model along with a MATLAB and Python wrapper around the pose estimator to easily explore pose estimation in other settings (Github link). The package contains precompiled Mac OS X 64-bit binaries so it should work out of the box on a modern Mac.
git clone https://github.com/stitchfix/PoseEstimation.git cd PoseEstimation python setup.py develop # Use sudo if necessary
Simply start MATLAB from the PoseEstimation folder to load the startup file. Alternatively, just call
demo in MATLAB for an example:
You can also make a new pose object manually and visualize it.
im = imread(fname); pose = poseest.Pose(im); pose_img = pose.show('image',im);
pose.show('image',im) displays the joints overlayed on the original image and returns the image matrix.
pose.show() just displays the connected joints as an image.
poseest.Pose object has the following properties:
pose = Pose with properties: EDGES: [1x1 struct] LIMBS: [1x1 struct] head_x: 131 head_y: 32 neck_x: 131 neck_y: 81 right_shoulder_x: 81 right_shoulder_y: 98 right_elbow_x: 81 right_elbow_y: 147 right_hand_x: 81 right_hand_y: 213 left_shoulder_x: 163 left_shoulder_y: 98 left_elbow_x: 180 left_elbow_y: 180 left_hand_x: 213 left_hand_y: 246 right_hip_x: 98 right_hip_y: 246 left_hip_x: 147 left_hip_y: 246 right_knee_x: 114 right_knee_y: 344 left_knee_x: 147 left_knee_y: 344 right_ankle_x: 131 right_ankle_y: 459 left_ankle_x: 147 left_ankle_y: 426 score: 0
This contains the
(x,y) coordinates of each feature point.
Examples of calling the pose estimator from Python can be found in the IPython Notebook.
import os from PoseEstimation import PoseEstimation root_path = os.path.abspath('./') # path to PoseEstimation/lib folder imgpath = os.path.join(root_path, "data/images") # path to folder of images image_paths = [os.path.join(imgpath,fn) for fn in next(os.walk(imgpath)) if 'jpg' in fn or 'png' in fn] pe = PoseEstimation.PoseEstimation(image_paths, root_path) labels,locations = pe.batch_estimate() # Start MATLAB once and compute poses on all images