Action Recognition Using Ballistic Dynamics

Shiv N. Vitaladevuni, Vili Kellokumpu and Larry S. Davis

Supplementary recognition results:

  1. Annotation of movements with "ballistic verbs".
  2. Gesture recognition.
  3. Tennis video.
  4. Dance video.
  5. Table assembly video.
  6. Carpentry workshop video.
  7. INRIA XMAS Dataset.


Annotation of movements with "ballistic verbs"

Two example videos are shown - one for reach movements and another for strike movements. The recognition result for each movement sequence is displayed as a video clip. First, the entire movement sequence is played. This is followed by a replay in which there is a pause at the start and end of each ballistic segment, and the recognition labels are displayed on the video frames.

Click on the images to play videos:

Click on the images to play videos:

Frame at start of ballistic segment shows the subject's silhouette and the head's gaze-direction. A message indicates that the video has been momentarily paused.

Recognition label for each ballistic segment are displayed during the intermediate frames of the segment.

Last frame for ballistic segment shows the recognition label for the segment, and the hand's target location in the image (marked with a red ellipse). A message is displayed at the top to indicate that the video has been momentarily paused.

Subject's gaze direction is indicated by yellow polygonal icon.

<< Back to top


Gesture Recognition Results

Dataset consists of 14 army signaling gestures [Shet04]. Each gesture is performed 5 times by 5 subjects. The ballistic action recognition framework using Motion History Image features achieved classification rate of 92%, compared to previously published rate of 84%. Recognition using MHI features without ballistic segmentation gave 73% accuracy. This highlights the utility and consistency of the ballistic segmentation.

Classification Results

<< Back to top


Tennis Video Example

Tennis video taken from [Efros et al. '03]. The video had 5 forehand swing shots, all were detected correctly. Animations of the ballistic segments obtained for 2 of the forehands:

Link to original video

(a.) Ballistic segments computed for forehand shot, duration frame id 285 to 329.
1. Frames 285 to 295

2. Frames 295 to 308

3. Frames 308 to 316

4. Frames 316 to 329

(b.) Ballistic segments computed for forehand shot, duration frame id 455 to 498.

1. Frames 455 to 464

2. Frames 464 to 470

3. Frames 470 to 484

4. Frames 484 to 498

<< Back to top


Dance Video Example

Tutorial video for a dance called "Grapevine Pop", courtesy www.fitmoves.com, duration 30 seconds, ~790 frames. The tutor performs 4 cycles of dance steps; first to her right, then a symmetric set of steps to her left, followed by another cycle to her right and one more to her left. Animations of the ballistic segments computed by the approach are shown here, each row corresponds to a cycle. The animated gifs are synchronized according to the movement. The consistency of the computed segments indicates the appropriateness of ballistic dynamics for the analysis. In the second cycle, segment boundary is missed because the tutor jumps almost out of the frame; this affected the optical flow computation.

Click here to view the original video.

<< Back to top