Visual Computing

University of Konstanz
Journal of Neuroscience Methods

Automated tracking and analysis of behavior in restrained insects

M. Shen, P. Szyszka, O. Deussen, C. Galizia, D. Merhof


Paper (.pdf, 3.2MB)


Background: Insect behavior is often monitored by human observers and measured in the form of binary responses. This procedure is time costly and does not allow a fine graded measurement of behavioral performance in individual animals. To overcome this limitation, we have developed a computer vision system which allows the automated tracking of body parts of restrained insects. New method: Our system crops a continuous video into separate shots with a static background. It then segments outthe insect’s head and preprocesses the detected moving objects to exclude detection errors. A Bayesian-based algorithm is proposed to identify the trajectory of each body part. Results: We demonstrate the application of this novel tracking algorithm by monitoring movements of the mouthparts and antennae of honey bees and ants, and demonstrate its suitability for analyzing the behavioral performance of individual bees using a common associative learning paradigm. Comparison with existing methods: Our tracking system differs from existing systems in that it does not require each video to be labeled manually and is capable of tracking insects’ body parts even when working with low frame-rate videos. Our system can be generalized for other insecttracking applications. Conclusions: Our system paves the ground for fully automated monitoring of the behavior of restrained insects and accounts for individual variations in graded behavior.


  author    = {M. Shen, P. Szyszka, O. Deussen, C. Galizia, D. Merhof},
  doi       = {10.1016/j.jneumeth.2014.10.021},
  issn      = {0165-0270},
  journal   = {Journal of Neuroscience Methods},
  keywords  = {Insect, Behavior, Honey bee, Classical conditioning, Multi-target tracking, Antenna},
  pages     = {194--205},
  title     = {Automated tracking and analysis of behavior in restrained insects},
  url       = {},
  volume    = {239},
  year      = {2015}