EyeMusic: Introducing a 'visual' colorful experience for the blind using auditory sensory substitution

Sami Abboud, Shlomi Hanassy, Shelly Levy-Tzedek, Shachar Maidenbaum, Amir Amedi

Research output: Contribution to journalArticlepeer-review

108 Scopus citations

Abstract

Purpose: Sensory-substitution devices (SSDs) provide auditory or tactile representations of visual information. These devices often generate unpleasant sensations and mostly lack color information. We present here a novel SSD aimed at addressing these issues. Methods: We developed the EyeMusic, a novel visual-to-auditory SSD for the blind, providing both shape and color information. Our design uses musical notes on a pentatonic scale generated by natural instruments to convey the visual information in a pleasant manner. A short behavioral protocol was utilized to train the blind to extract shape and color information, and test their acquired abilities. Finally, we conducted a survey and a comparison task to assess the pleasantness of the generated auditory stimuli. Results: We show that basic shape and color information can be decoded from the generated auditory stimuli. High performance levels were achieved by all participants following as little as 2-3 hours of training. Furthermore, we show that users indeed found the stimuli pleasant and potentially tolerable for prolonged use. Conclusions: The novel EyeMusic algorithm provides an intuitive and relatively pleasant way for the blind to extract shape and color information. We suggest that this might help facilitating visual rehabilitation because of the added functionality and enhanced pleasantness.

Original languageEnglish
Pages (from-to)247-257
Number of pages11
JournalRestorative Neurology and Neuroscience
Volume32
Issue number2
DOIs
StatePublished - 1 Jan 2014
Externally publishedYes

Keywords

  • Blindness
  • Color
  • Pleasant sounds
  • Rehabilitation
  • SSD
  • Sensory substitution devices

Fingerprint

Dive into the research topics of 'EyeMusic: Introducing a 'visual' colorful experience for the blind using auditory sensory substitution'. Together they form a unique fingerprint.

Cite this