 |
    |
Pipelining Localized Semantic Features for Fine-Grained Action Recognition
Yang Zhou1, Bingbing Ni2, Shuicheng Yan3, Pierre Moulin4, and Qi Tian1
1University of Texas at San Antonio, USA
myh511@my.utsa.edu
qi.tian@utsa.edu
2Advanced Digital Sciences Center, Singapore
bingbing.ni@adsc.com.sg
3National University of Singapore, Singapore
eleyans@nus.edu.sg
4University of Illinois at Urbana-Champaign, USA
moulin@ifp.uiuc.edu
Abstract. In fine-grained action (object manipulation) recognition, it is important to encode object semantic (contextual) information, i.e., which object is being manipulated and how it is being operated. However, previous methods for action recognition often represent the semantic information in a global and coarse way and therefore cannot cope with fine-grained actions. In this work, we propose a representation and classification pipeline which seamlessly incorporates localized semantic information into every processing step for fine-grained action recognition. In the feature extraction stage, we explore the geometric information between local motion features and the surrounding objects. In the feature encoding stage, we develop a semantic-grouped locality-constrained linear coding (SG-LLC) method that captures the joint distributions between motion and object-in-use information. Finally, we propose a semantic-aware multiple kernel learning framework (SA-MKL) by utilizing the empirical joint distribution between action and object type for more discriminative action classification. Extensive experiments are performed on the large-scale and difficult fine-grained MPII cooking action dataset. The results show that by effectively accumulating localized semantic information into the action representation and classification pipeline, we significantly improve the fine-grained action classification performance over the existing methods.
LNCS 8692, p. 481 ff.
lncs@springer.com
© Springer International Publishing Switzerland 2014