State-machine based approach for improving robustness in multimodal control

H. Stern; E. Blanc-Baron

doi:10.1109/VECIMS.2003.1227029

State-machine based approach for improving robustness in multimodal control

H. Stern, E. Blanc-Baron

Department of Industrial Engineering and Management

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Multimodal systems are aimed to overcome the disadvantages of unimodal systems, especially the lack of robustness. In this paper, a new approach to modeling multimodal systems, by dividing the system to independent state machines, is proffered. The state machine approach is supposed to be more intuitive to the user and to increase redundancy, especially in noisy environments. Intuitiveness is achieved by adapting each state machine to the user's expectations. Redundancy is achieved by allowing the user to use more than one modality to express the same utterance. Its task-oriented grammar allows the reuse of identical commands thus reducing the overall size of command set. The architecture also allows increased recognition accuracy by decomposing the large command vocabulary into smaller subsets. The system has been tested on a small command vocabulary in a simulated environment. The results of the comparison between pure gesture and pure voice showed that the bimodal system was superior.

Original language	English
Title of host publication	VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems
Publisher	Institute of Electrical and Electronics Engineers
Pages	51-55
Number of pages	5
ISBN (Electronic)	0780377850, 9780780377851
DOIs	https://doi.org/10.1109/VECIMS.2003.1227029
State	Published - 1 Jan 2003
Event	2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems, VECIMS 2003 - Lugano, Switzerland Duration: 27 Jul 2003 → 29 Jul 2003

Publication series

Name	VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems

Conference

Conference	2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems, VECIMS 2003
Country/Territory	Switzerland
City	Lugano
Period	27/07/03 → 29/07/03

Keywords

Acoustical engineering
Additive noise
Character recognition
Focusing
Noise robustness
Redundancy
Robust control
Speech recognition
Vocabulary
Working environment noise

ASJC Scopus subject areas

Artificial Intelligence
Human-Computer Interaction
Control and Systems Engineering
Electrical and Electronic Engineering

Access to Document

10.1109/VECIMS.2003.1227029

Cite this

Stern, H., & Blanc-Baron, E. (2003). State-machine based approach for improving robustness in multimodal control. In VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems (pp. 51-55). [1227029] (VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/VECIMS.2003.1227029

Stern, H. ; Blanc-Baron, E. / State-machine based approach for improving robustness in multimodal control. VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems. Institute of Electrical and Electronics Engineers, 2003. pp. 51-55 (VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems).

@inproceedings{51b80ddd92b5431990ac010f66088211,

title = "State-machine based approach for improving robustness in multimodal control",

abstract = "Multimodal systems are aimed to overcome the disadvantages of unimodal systems, especially the lack of robustness. In this paper, a new approach to modeling multimodal systems, by dividing the system to independent state machines, is proffered. The state machine approach is supposed to be more intuitive to the user and to increase redundancy, especially in noisy environments. Intuitiveness is achieved by adapting each state machine to the user's expectations. Redundancy is achieved by allowing the user to use more than one modality to express the same utterance. Its task-oriented grammar allows the reuse of identical commands thus reducing the overall size of command set. The architecture also allows increased recognition accuracy by decomposing the large command vocabulary into smaller subsets. The system has been tested on a small command vocabulary in a simulated environment. The results of the comparison between pure gesture and pure voice showed that the bimodal system was superior.",

keywords = "Acoustical engineering, Additive noise, Character recognition, Focusing, Noise robustness, Redundancy, Robust control, Speech recognition, Vocabulary, Working environment noise",

author = "H. Stern and E. Blanc-Baron",

note = "Publisher Copyright: {\textcopyright} 2003 IEEE.; 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems, VECIMS 2003 ; Conference date: 27-07-2003 Through 29-07-2003",

year = "2003",

month = jan,

day = "1",

doi = "10.1109/VECIMS.2003.1227029",

language = "English",

series = "VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems",

publisher = "Institute of Electrical and Electronics Engineers",

pages = "51--55",

booktitle = "VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems",

address = "United States",

}

Stern, H & Blanc-Baron, E 2003, State-machine based approach for improving robustness in multimodal control. in VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems., 1227029, VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems, Institute of Electrical and Electronics Engineers, pp. 51-55, 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems, VECIMS 2003, Lugano, Switzerland, 27/07/03. https://doi.org/10.1109/VECIMS.2003.1227029

State-machine based approach for improving robustness in multimodal control. / Stern, H.; Blanc-Baron, E.

VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems. Institute of Electrical and Electronics Engineers, 2003. p. 51-55 1227029 (VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - State-machine based approach for improving robustness in multimodal control

AU - Stern, H.

AU - Blanc-Baron, E.

PY - 2003/1/1

Y1 - 2003/1/1

N2 - Multimodal systems are aimed to overcome the disadvantages of unimodal systems, especially the lack of robustness. In this paper, a new approach to modeling multimodal systems, by dividing the system to independent state machines, is proffered. The state machine approach is supposed to be more intuitive to the user and to increase redundancy, especially in noisy environments. Intuitiveness is achieved by adapting each state machine to the user's expectations. Redundancy is achieved by allowing the user to use more than one modality to express the same utterance. Its task-oriented grammar allows the reuse of identical commands thus reducing the overall size of command set. The architecture also allows increased recognition accuracy by decomposing the large command vocabulary into smaller subsets. The system has been tested on a small command vocabulary in a simulated environment. The results of the comparison between pure gesture and pure voice showed that the bimodal system was superior.

AB - Multimodal systems are aimed to overcome the disadvantages of unimodal systems, especially the lack of robustness. In this paper, a new approach to modeling multimodal systems, by dividing the system to independent state machines, is proffered. The state machine approach is supposed to be more intuitive to the user and to increase redundancy, especially in noisy environments. Intuitiveness is achieved by adapting each state machine to the user's expectations. Redundancy is achieved by allowing the user to use more than one modality to express the same utterance. Its task-oriented grammar allows the reuse of identical commands thus reducing the overall size of command set. The architecture also allows increased recognition accuracy by decomposing the large command vocabulary into smaller subsets. The system has been tested on a small command vocabulary in a simulated environment. The results of the comparison between pure gesture and pure voice showed that the bimodal system was superior.

KW - Acoustical engineering

KW - Additive noise

KW - Character recognition

KW - Focusing

KW - Noise robustness

KW - Redundancy

KW - Robust control

KW - Speech recognition

KW - Vocabulary

KW - Working environment noise

UR - http://www.scopus.com/inward/record.url?scp=84943595987&partnerID=8YFLogxK

U2 - 10.1109/VECIMS.2003.1227029

DO - 10.1109/VECIMS.2003.1227029

M3 - Conference contribution

AN - SCOPUS:84943595987

T3 - VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems

SP - 51

EP - 55

BT - VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems

PB - Institute of Electrical and Electronics Engineers

T2 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems, VECIMS 2003

Y2 - 27 July 2003 through 29 July 2003

ER -

Stern H, Blanc-Baron E. State-machine based approach for improving robustness in multimodal control. In VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems. Institute of Electrical and Electronics Engineers. 2003. p. 51-55. 1227029. (VECIMS 2003 - 2003 International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems). doi: 10.1109/VECIMS.2003.1227029

State-machine based approach for improving robustness in multimodal control

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this