A Declarative Scale-, Rotation- and Position-invariant 3D Gesture Algorithm

Student Name
Ruben Tytgat
Thesis Type
Bachelor Thesis
Thesis Status
Finished
Academic Year
2012 - 2013
Degree
Bachelor in de Computerwetenschappen
Promoter
Beat Signer
Supervisor(s)
Download
Description

Current gesture recognition approaches typically struggle to accurately detect gestures in real-time. Most gesture recognition approaches require prior segmentation in order to be tractable. In earlier work by Lode Hoste he tried to tackle this problem by using a declarative language and a RETE engine to detect whether points of a trajectory correspond to certain constraints. Thanks to the caching of intermediate calculations in the RETE engine, this approach allows to detect gestures at real-time. Currently, the approach allows to define gestures which are position invariant but scale and rotational invariant gestures are still very computational expensive especially when applied to 3D trajectories.

Goal:

In this project the goal is to design an algorithm based on different features and a divide and conquer strategy to detect gestures regardless of the scale, position or rotation of the gestures.

 

This thesis consists of several parts:

  1. Start with a few control points on a gesture in 2D (considering only x and y-axis).
  2. Think about possible constraints that are efficient and filter out a lot of potential matches at the same time.
  3. Implement these constraints first manually and check how many potential matches they filter out, try to filter out as many matches as possible with the least calculations possible.
  4. (Optional) When a tractable algorithm is tested and seems to work, implement the code generation to automatically derive this declarative algorithm from a set of control points.
  5. (Optional) Test the performance of this algorithm on several samples. It's important to know the amount of non-detected gestures, the amount of false-positives, the amount of gestures that can run at the same time in a real-time application given that there are 30 points per second generated. 
  6. (Optional) Extend to 3D.
  7. (Optional) Automatically generate the control points.

The implemention is done in C# and uses the Mudra engine. Voltra is used to see the trajectories and place the control points and is extended to generate new code for the scale and rotation invariant gestures.