DeepPoseKit
An open-source Python toolkit for 2D animal pose estimation, with TensorFlow and Keras models, image annotation, and GPU-assisted keypoint detection.
DeepPoseKit is an open-source Python toolkit for researchers measuring animal movement and behavior in images or video. You define the body points you want to track and train a model on your own annotations. It runs through TensorFlow and Keras on your own hardware, with Windows support documented, and uses the Apache 2.0 license.
The toolkit covers annotation, model training and prediction on new footage. Its interactive annotation tool lets researchers label points on images or video frames, while imgaug provides data augmentation to account for variation in the footage. The Keras-based interface supports training, evaluation, and saving or loading models.
Model choices include Stacked DenseNet and Stacked Hourglass, along with DeepLabCut and LEAP architectures. Stacked DenseNet uses a multi-scale architecture for keypoint estimation. GPU-accelerated peak detection locates points with subpixel precision. The research includes tests on challenging animal datasets and interacting groups in the wild, with a peer-reviewed paper in eLife.
DeepPoseKit focuses on individual pose estimation. Visually distinct animals can have separate sets of labeled points within the same image. For animals that look alike, workflows can combine it with separate localization and tracking software such as idtracker.ai or Tracktor to isolate each animal before estimating its pose.
