Temporal characteristics of audiovisual information processing

Galit Fuhrmann Alpert, Grit Hein, Nancy Tsai, Marcus J. Naumer, Robert T. Knight

Research output: Contribution to journalArticlepeer-review

36 Scopus citations

Abstract

In complex natural environments, auditory and visual information often have to be processed simultaneously. Previous functional magnetic resonance imaging (fMRI) studies focused on the spatial localization of brain areas involved in audiovisual (AV) information processing, but the temporal characteristics of AV information flow in these regions remained unclear. In this study, we used fMRI and a novel information-theoretic approach to study the flow of AV sensory information. Subjects passively perceived sounds and images of objects presented either alone or simultaneously. Applying the measure of mutual information, we computed for each voxel the latency in which the blood oxygenation level-dependent signal had the highest information content about the preceding stimulus. The results indicate that, after AV stimulation, the earliest informative activity occurs in right Heschl's gyrus, left primary visual cortex, and the posterior portion of the superior temporal gyrus, which is known as a region involved in object-related AV integration. Informative activity in the anterior portion of superior temporal gyrus, middle temporal gyrus, right occipital cortex, and inferior frontal cortex was found at a later latency. Moreover, AV presentation resulted in shorter latencies in multiple cortical areas compared with isolated auditory or visual presentation. The results provide evidence for bottom-up processing from primary sensory areas into higher association areas during AV integration in humans and suggest that AV presentation shortens processing time in early sensory cortices.

Original languageEnglish
Pages (from-to)5344-5349
Number of pages6
JournalJournal of Neuroscience
Volume28
Issue number20
DOIs
StatePublished - 14 May 2008
Externally publishedYes

Keywords

  • Auditory
  • fMRI
  • Information theory
  • Multisensory
  • Prefrontal cortex
  • Visual

ASJC Scopus subject areas

  • Neuroscience (all)

Fingerprint

Dive into the research topics of 'Temporal characteristics of audiovisual information processing'. Together they form a unique fingerprint.

Cite this