Multi-touch technology allows users to use their hands to manipulate digital information. We have observed that mainstream software frameworks do not offer support to deal with the complexity of these new devices. Current multi-touch frameworks only provide a narrow range of hardcoded functionality. Therefore, the development of new multi-touch gestures and their integration with other gestures is notoriously hard. The main goal of the Midas framework is to provide developers adequate software engineering abstractions to close the gap between the evolution in the multi-touch technology and software detection mechanisms.

Current frameworks force the programmer into an event driven programming model where the programmer has to register and compose event handlers manually. This results in applications where the control flow of the application is driven by external events and no longer by the sequential structure of the program. The reuse, composition and understanding of program code gets difficult when using such frameworks.

In this work, we propose a solution based on research conducted in the complex event processing domain. We advocate the use of a rule language which allows programmers to express gestures in a declarative way. The advantage of such an approach is that the programmer no longer needs to be concerned about how to derive gestures but only about describing the gesture. We present a first step in that direction in the form of a domain-specific language supporting spatio-temporal operators.

Complex gestures which are extremely hard to be implemented in traditional approaches can be expressed in one or multiple rules which are easy to understand. The use of a rule language has the benefit that the developed gestures are reusable and easy to compose. Further, a strong connection to application-level entities allows developers to activate and deactivate gestures depending on their graphical context.

Related Publications