Medical Image Analysis

ASL Recognition

The first video demonstrats detection of a wh-question non-manual marker. Tracked face and head is shown on left, while the right image shows the extracted spatial pyramid features. Red bars indicate detection of the wh non-manual marker, while blue bars indicate that the system detects no wh non-manual marker.
In the second video we demonstrate that we are able to track eyebrow height (top right) and head pitch angle (bottom right) in an isolated utterance of wh-question. The red graph line is used to identify the segment of the sequence over which the nonmanual marker is being produced. Lowered eyebrows correspond to a wh-question marker being present.

Collaboration with: 

National Center for Sign Language and Gesture Resources



  • Dimitris Metaxas, Bo Liu, Fei Yang, Peng Yang, Nicholas Michael and Carol Neidle: "Recognition of Nonmanual Markers in American Sign Language (ASL) Using Non-Parametric Adaptive 2D-3D Face Tracking". LREC, 2012.
  • Vogler, C., Metaxas, D.: "ASL recognition based on a coupling between HMMs and 3D motion analysis". ICCV, 1998