Detecting Social Actions of Fruit Flies^*

Eyrun Eyjolfsdottir¹, Steve Branson¹, Xavier P. Burgos-Artizzu¹, Eric D. Hoopfer¹, Jonathan Schor¹, David J. Anderson^1,2, and Pietro Perona¹

¹California Institute of Technology, Pasadena, California, USA

²Howard Hughes Medical Institute (HHMI), USA

Abstract. We describe a system that tracks pairs of fruit flies and automatically detects and classifies their actions. We compare experimentally the value of a frame-level feature representation with the more elaborate notion of ‘bout features’ that capture the structure within actions. Similarly, we compare a simple sliding window classifier architecture with a more sophisticated structured output architecture, and find that window based detectors outperform the much slower structured counterparts, and approach human performance. In addition we test our top performing detector on the CRIM13 mouse dataset, finding that it matches the performance of the best published method. Our Fly-vs-Fly dataset contains 22 hours of video showing pairs of fruit flies engaging in 10 social interactions in three different contexts; it is fully annotated by experts, and published with articulated pose trajectory features.

Electronic Supplementary Material:

• Electronic Supplementary Material (PDF 2,777 KB)

LNCS 8690, p. 772 ff.

Full article in PDF | BibTeX

lncs@springer.com
© Springer International Publishing Switzerland 2014