"A gesture is a form of non-verbal communication in which visible bodily actions communicate particular messages, either in place of speech or together and in parallel with spoken words" [1]. Recently we have developed Midas [2], a declarative multi-touch gesture recogniser which is capable of recognising advanced gestural interaction. We are currently extending this approach to detect gestures using 3D cameras, such as the Microsoft Xbox Kinect camera. Although gestures can be implemented with great ease based on our approach, it is sometimes difficult to know why a gesture fails to trigger. This is usually related to finding the correct spatial and temporal values (e.g. x pixels between the movement of fingers on multi-touch surfaces within y milliseconds or 3D coordinates for a the Kinect camera).

The goal of this thesis is to build a 3D gesture debugger. This is a very open topic since there has not been a lot of research and development in this area. What we envision is a graphical editor where spatial and temporal relations can be expressed and tested. For example, we could record the movement of a user via the Kinect camera and afterwards the editor should be able to replay this movement frame by frame. Instead of having to specify spatial and temporal constraints manually, a graphical annotation service could be offered to the developer.

At the end of the thesis, we envision to be able to design, develop and debug gestures in real-time using a 3D camera and a 3D screen. This means the avatar is presented on a 3D screen and developers can design and compile their customised gestures by grasping the 3D image in thin air (the kinect camera will capture the z/y/z coordinates of the hands with the necessary precision).

[1] http://en.wikipedia.org/wiki/Gesture

[2] Midas: A Declarative Multi-Touch Interaction Framework, Christophe Scholiers, Lode Hoste, Beat Signer and Wolfgang De Meuter, In Proceedings of the 5th International Conference on Tangible, Embedded and Embodied Interaction (TEI 2011), Funchal, Portugal, January 2011.