Eye and Voice-Controlled Human Machine Interface System for Wheelchairs Using Image Gradient Approach

Saba Anwer, Asim Waris, Hajrah Sultan, Shahid Ikramullah, Muhammad Hamza, Moaz Sarwar, Imran Khan Niazi, Muhammad Shafique, Amit N. Pujari

Research output: Contribution to journalJournal articleResearchpeer-review

Abstract

Rehabilitative mobility aids are being used extensively for physically impaired people. Efforts are being made to develop human machine interfaces (HMIs), manipulating the biosignals to better control the electromechanical mobility aids, especially the wheelchairs. Creating precise control commands such as move forward, left, right, backward and stop, via biosignals, in an appropriate HMI is the actual challenge, as the people with a high level of disability (quadriplegia and paralysis, etc.) are unable to drive conventional wheelchairs. Therefore, a novel system driven by optical signals addressing the needs of such a physically impaired population is introduced in this paper. The present system is divided into two parts: the first part comprises of detection of eyeball movements together with the processing of the optical signal, and the second part encompasses the mechanical assembly module, i.e., control of the wheelchair through motor driving circuitry. A web camera is used to capture real-time images. The processor used is Raspberry-Pi with Linux operating system. In order to make the system more congenial and reliable, the voice-controlled mode is incorporated in the wheelchair. To appraise the system’s performance, a basic wheelchair skill test (WST) is carried out. Basic skills like movement on plain and rough surfaces in forward, reverse direction and turning capability were analyzed for easier comparison with other existing wheelchair setups on the bases of controlling mechanisms, compatibility, design models, and usability in diverse conditions. System successfully operates with average response time of 3 s for eye and 3.4 s for voice control mode.
Original languageEnglish
Article number5510
JournalSensors
Volume20
Issue number19
Pages (from-to)1-13
Number of pages13
ISSN1424-8220
DOIs
Publication statusPublished - 26 Sep 2020

Keywords

  • AMR voice
  • Human machine interface (HMI)
  • Image gradient
  • Image processing
  • Open-CV
  • Quadriplegia
  • Raspberry Pi
  • Rehabilitation
  • Wheelchair

Fingerprint Dive into the research topics of 'Eye and Voice-Controlled Human Machine Interface System for Wheelchairs Using Image Gradient Approach'. Together they form a unique fingerprint.

  • Cite this

    Anwer, S., Waris, A., Sultan, H., Ikramullah, S., Hamza, M., Sarwar, M., Niazi, I. K., Shafique, M., & Pujari, A. N. (2020). Eye and Voice-Controlled Human Machine Interface System for Wheelchairs Using Image Gradient Approach. Sensors, 20(19), 1-13. [5510]. https://doi.org/10.3390/s20195510