Multisensory Visual-Auditory Object Recognition in Humans: A High-density Electrical Mapping Study

Sophie Molholm, Walter Ritter, Daniel C. Javitt, John J. Foxe

Research output: Contribution to journalArticle

250 Citations (Scopus)

Abstract

Multisensory object-recognition processes were investigated by examining the combined influence of visual and auditory inputs upon object identification - in this case, pictures and vocalizations of animals. Behaviorally, subjects were significantly faster and more accurate at identifying targets when the picture and vocalization were matched (i.e. from the same animal), than when the target was represented in only one sensory modality. This behavioral enhancement was accompanied by a modulation of the evoked potential in the latency range and general topographic region of the visual evoked N1 component, which is associated with early feature processing in the ventral visual stream. High-density topographic mapping and dipole modeling of this multisensory effect were consistent with generators in lateral occipito-temporal cortices, suggesting that auditory inputs were modulating processing in regions of the lateral occipital cortices. Both the timing and scalp topography of this modulation suggests that there are multisensory effects during what is considered to be a relatively early stage of visual object-recognition processes, and that this modulation occurs in regions of the visual system that have traditionally been held to be unisensory processing areas. Multisensory inputs also modulated the visual 'selection-negativity', an attention dependent component of the evoked potential this is usually evoked when subjects selectively attend to a particular feature of a visual stimulus.

Original languageEnglish (US)
Pages (from-to)452-465
Number of pages14
JournalCerebral Cortex
Volume14
Issue number4
DOIs
StatePublished - Apr 2004
Externally publishedYes

Fingerprint

Evoked Potentials
Animal Vocalization
Occipital Lobe
Temporal Lobe
Scalp

Keywords

  • Auditory
  • Electrical mapping
  • Humans
  • Multisensory
  • Object recognition
  • Visual

ASJC Scopus subject areas

  • Neuroscience(all)

Cite this

Multisensory Visual-Auditory Object Recognition in Humans : A High-density Electrical Mapping Study. / Molholm, Sophie; Ritter, Walter; Javitt, Daniel C.; Foxe, John J.

In: Cerebral Cortex, Vol. 14, No. 4, 04.2004, p. 452-465.

Research output: Contribution to journalArticle

Molholm, Sophie ; Ritter, Walter ; Javitt, Daniel C. ; Foxe, John J. / Multisensory Visual-Auditory Object Recognition in Humans : A High-density Electrical Mapping Study. In: Cerebral Cortex. 2004 ; Vol. 14, No. 4. pp. 452-465.
@article{9a8a222eec344eef9e0b06abde93b80c,
title = "Multisensory Visual-Auditory Object Recognition in Humans: A High-density Electrical Mapping Study",
abstract = "Multisensory object-recognition processes were investigated by examining the combined influence of visual and auditory inputs upon object identification - in this case, pictures and vocalizations of animals. Behaviorally, subjects were significantly faster and more accurate at identifying targets when the picture and vocalization were matched (i.e. from the same animal), than when the target was represented in only one sensory modality. This behavioral enhancement was accompanied by a modulation of the evoked potential in the latency range and general topographic region of the visual evoked N1 component, which is associated with early feature processing in the ventral visual stream. High-density topographic mapping and dipole modeling of this multisensory effect were consistent with generators in lateral occipito-temporal cortices, suggesting that auditory inputs were modulating processing in regions of the lateral occipital cortices. Both the timing and scalp topography of this modulation suggests that there are multisensory effects during what is considered to be a relatively early stage of visual object-recognition processes, and that this modulation occurs in regions of the visual system that have traditionally been held to be unisensory processing areas. Multisensory inputs also modulated the visual 'selection-negativity', an attention dependent component of the evoked potential this is usually evoked when subjects selectively attend to a particular feature of a visual stimulus.",
keywords = "Auditory, Electrical mapping, Humans, Multisensory, Object recognition, Visual",
author = "Sophie Molholm and Walter Ritter and Javitt, {Daniel C.} and Foxe, {John J.}",
year = "2004",
month = "4",
doi = "10.1093/cercor/bhh007",
language = "English (US)",
volume = "14",
pages = "452--465",
journal = "Cerebral Cortex",
issn = "1047-3211",
publisher = "Oxford University Press",
number = "4",

}

TY - JOUR

T1 - Multisensory Visual-Auditory Object Recognition in Humans

T2 - A High-density Electrical Mapping Study

AU - Molholm, Sophie

AU - Ritter, Walter

AU - Javitt, Daniel C.

AU - Foxe, John J.

PY - 2004/4

Y1 - 2004/4

N2 - Multisensory object-recognition processes were investigated by examining the combined influence of visual and auditory inputs upon object identification - in this case, pictures and vocalizations of animals. Behaviorally, subjects were significantly faster and more accurate at identifying targets when the picture and vocalization were matched (i.e. from the same animal), than when the target was represented in only one sensory modality. This behavioral enhancement was accompanied by a modulation of the evoked potential in the latency range and general topographic region of the visual evoked N1 component, which is associated with early feature processing in the ventral visual stream. High-density topographic mapping and dipole modeling of this multisensory effect were consistent with generators in lateral occipito-temporal cortices, suggesting that auditory inputs were modulating processing in regions of the lateral occipital cortices. Both the timing and scalp topography of this modulation suggests that there are multisensory effects during what is considered to be a relatively early stage of visual object-recognition processes, and that this modulation occurs in regions of the visual system that have traditionally been held to be unisensory processing areas. Multisensory inputs also modulated the visual 'selection-negativity', an attention dependent component of the evoked potential this is usually evoked when subjects selectively attend to a particular feature of a visual stimulus.

AB - Multisensory object-recognition processes were investigated by examining the combined influence of visual and auditory inputs upon object identification - in this case, pictures and vocalizations of animals. Behaviorally, subjects were significantly faster and more accurate at identifying targets when the picture and vocalization were matched (i.e. from the same animal), than when the target was represented in only one sensory modality. This behavioral enhancement was accompanied by a modulation of the evoked potential in the latency range and general topographic region of the visual evoked N1 component, which is associated with early feature processing in the ventral visual stream. High-density topographic mapping and dipole modeling of this multisensory effect were consistent with generators in lateral occipito-temporal cortices, suggesting that auditory inputs were modulating processing in regions of the lateral occipital cortices. Both the timing and scalp topography of this modulation suggests that there are multisensory effects during what is considered to be a relatively early stage of visual object-recognition processes, and that this modulation occurs in regions of the visual system that have traditionally been held to be unisensory processing areas. Multisensory inputs also modulated the visual 'selection-negativity', an attention dependent component of the evoked potential this is usually evoked when subjects selectively attend to a particular feature of a visual stimulus.

KW - Auditory

KW - Electrical mapping

KW - Humans

KW - Multisensory

KW - Object recognition

KW - Visual

UR - http://www.scopus.com/inward/record.url?scp=1942518364&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=1942518364&partnerID=8YFLogxK

U2 - 10.1093/cercor/bhh007

DO - 10.1093/cercor/bhh007

M3 - Article

C2 - 15028649

AN - SCOPUS:1942518364

VL - 14

SP - 452

EP - 465

JO - Cerebral Cortex

JF - Cerebral Cortex

SN - 1047-3211

IS - 4

ER -