Putting out a Helping Hand to Machine Learning - Gesture Template Fusion

Type of Thesis: 
Master Thesis


Gesture recognition engines rely on machine learning techniques to learn and detect gestures. Algorithms such as Dynamic Time Warping [3, 4], Linear Time Warping [4], One Dollar recogniser [2], Rubine (3D) [1], SiGeR [1], Support Vector Machines [3], Neural Networks [3], and Hidden Markov Models (HMM) [5, 6] are well known in this domain. Although each recogniser claims to be a complete solution for gesture detection problems, we experienced that some algorithms either provide false positives very easily while others are more reserved. (For example: open http://depts.washington.edu/aimgroup/proj/dollar/ and try to draw a clockwise circle).

In this thesis, we plan to fuse different machine learning techniques to improve their gesture recognition rate. This means we are going to use two or three different recognisers at the same data and describe in what situation we can achieve a higher fidelity ratio.

The thesis contains the following activities:
  1. Port existing recognisers to Midas (programmed in C)
  2. Find a way to express the fusion mechanism (domain-specific language, ...)
  3. Perform an evaluation and show which techniques work best depending on the properties of the gesture and what combination of algorithms generally provides the best results.
(An example of a false positive with more than 60% certainty)
[1] iGesture: A General Gesture Recognition Framework. Beat Signer, Ueli Kurmann and Moira C. Norrie, Proceedings of ICDAR 2007, 9th International Conference on Document Analysis and Recognition, Curitiba, Brazil, September 2007.
[2] Wobbrock, J.O., Wilson, A.D. and Li, Y. (2007). Gestures Without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes. Proceedings of the ACM Symposium on User Interface Software and Technology (UIST 2007). Newport, Rhode Island (October 7-10, 2007). New York: ACM Press, pp. 159-168.
[3] Robert NeBelrath and Jan Alexandersson. A 3D Gesture Recognition System for Multimodal Dialog Systems. In Proceedings of IJCAI 2009, 6th IJCAI Workshop on Knowledge and Reasoning in Practical Dialogue Systems, pages 46-51, Pasadena, USA, July 2009.
[4] Daniel Wilson and Andy Wilson. Gesture Recognition Using the XWand. Technical Report CMU-RI-TR-04-57, Carnegie Mellon University, Pittsburgh, PA, April 2004.
[5] A. D. Wilson and A. F. Bobick. Parametric Hidden Markov Models for Gesture Recognition. IEEE Trans. Pattern Anal. Mach. Intell., 21(9):884-900, 1999.
[6] J. Yang and Y. Xu. Hidden Markov Model for Gesture Recognition. Technical Report CMU-RI-TR-94-10, Robotics Institute, Pittsburgh, PA, May 1994.
Background Knowledge: 
  • C programming experience
  • Machine Learning


Technical challenges: 
  • Pattern recognition
  • Gestural interaction
Beat Signer
Bruno Dumas
Lode Hoste
Academic Year: