Improving Error Correction and Text Editing Using Voice and Mouse Multimodal Interface

Meirav Taieb-Maimon, Luiza Romanovskii-Chernik

Research output: Contribution to journalArticlepeer-review


Given the widespread use of word processing systems today, error correction and text editing tasks are frequently performed. A novel method for error correction and text editing using voice&mouse was developed and evaluated. A within-participants counterbalanced study design was used to compare users’ performance and satisfaction with the proposed method to their performance and satisfaction with two other methods that enable natural interaction with Microsoft Word, voice-only and voice&gaze, and with conventional interaction using the keyboard&mouse method, when performing error correction and text editing tasks. Our comprehensive evaluation showed that for all of the examined tasks the proposed voice&mouse method performed comparably to the voice-only and voice&gaze methods or significantly outperformed them on all evaluation measures. The keyboard&mouse method yielded significantly higher average SUS scores and lower NASA-TLX task loads than the other methods. However, participants significantly preferred using the voice&mouse method, in general, and especially for tasks that typically require typing in addition to using a mouse, such as fix, replace, and add, and the performance times for the two methods were comparable when performing these tasks. The findings support the use of the voice&mouse method for error correction and text editing along with the keyboard&mouse method. This would allow users to choose their preferred interaction method for each scenario and task while improving their performance and satisfaction.

Original languageAmerican English
JournalInternational Journal of Human-Computer Interaction
StateAccepted/In press - 1 Jan 2024


  • eye tracking
  • interaction design
  • Multimodal interfaces
  • voice

All Science Journal Classification (ASJC) codes

  • Human Factors and Ergonomics
  • Human-Computer Interaction
  • Computer Science Applications

Cite this