Gesture Recognition

This video is from a practice sessions for a live demonstration given at the Communicating with Computers program in May, 2017. It shows a person (Rahul, in Fort Collins CO) directing an avatar (Diana, in Waltham MA) to build block structures (in this case, a staircase from three blocks). Rahul is only allowed to gesture; Diana is allowed to gesture, move blocks, and ask questions through text. The demonstration was joint work with James Pustejovsky and his lab at Brandeis (who created Diana and her reasoning system), and Jaime Ruiz and his lab at the University of Florida (who elicited the gesture set from naive users). CSU built the real-time gesture recognition system.

More Communication through Gestures, Expressions and Shared Perception

Action Recognition

Play the video below to see an example of earlier work in action recognition. This was joint work in the summer of 2012 with iRobot and U.C. Berkeley as part of the DARPA Mind's Eye project. (Many thanks to Dr. Christopher Geyer and the folks at iRobot for producing this video).

More Mind's Eye