Fusion in Multimodal Interactive Systems: An HMM-based Algorithm for User-Induced Adaptation
Multimodal interfaces have shown to be ideal candidates for interactive systems that adapt to a user either automatically or based on user-defined rules. However, user-based adaptation demands for the corresponding advanced software architectures and algorithms. We present a novel multimodal fusion algorithm for the development of adaptive interactive systems which is based on hidden Markov models~(HMM). In order to select relevant modalities at the semantic level, the algorithm is linked to temporal relationship properties. The presented algorithm has been evaluated in three use cases from which we were able to identify the main challenges involved in developing adaptive multimodal interfaces.
Publication Reference
Dumas, B., Signer, B. and Lalanne, D.: "Fusion in Multimodal Interactive Systems: An HMM-Based Algorithm for User Induced Adaptation", Proceedings of the 4th International Conference on Engineering Interactive Computing Systems (EICS 2012), Copenhagen, Denmark, June 2012
Available:
