Neurophysiology of Multisensory Integration

DESCRIPTION (provided by applicant): That the different senses sample unique aspects of physical objects should provide the brain with both a richer description of objects, and converging evidence concerning their position, identity and movement. Although there has been a detailed explication of multi sensory processing in the superior colliculus of cats and a number of multi sensory brain regions have been detailed in non-human primates, we have only a rudimentary understanding of how information from different sensory systems is combined in the human neo-cortex. The purpose of this proposal is to further our understanding of human cortical multi sensory integration through the combined use of event-related potential (ERP) recordings and functional imaging (fMRI). High-density mapping of ERPs recorded from 128 scalp electrodes, co-registration of these maps with individual subject MRI-derived anatomy, and source analysis of the surface recorded data will allow us to assess the brain regions involved in such integrations. Parallel fMRI experiments will allow us to precisely define the brain regions involved in a given multi sensory operation. Integration of ERP source-analyses and fMRI data will provide the spatio-temporal dynamics of multi sensory integration. Critically, the precise temporal resolution of the ERP will allow us to assess the time-course of multi sensory integration effects relative to the time-course of ongoing unisensory processing. We propose to determine: 1) the temporal separation parameters between the unisensory constituents of a bi-sensory audio-visual stimulus that result in modifications of the ERP interactions associated with bi-sensory stimulation, 2) whether spatial separation of bi-sensory stimuli modifies the early ERP interactions associated with bi-sensory stimulation, 3) whether top-down influences (attention) can effect the earliest multi-sensory interactions to bi-sensory stimulation, 4) to assess the effects of simultaneous stimulation in an entirely task-irrelevant sensory modality (vision) upon the ERP interactions with a second task-relevant sensory modality (somesthesis). Specifically, we wish to assess the effects of the irrelevant modality when it either contradicts or confirms task-relevant spatial information, 5) whether illusory changes in the perception of visual apparent motion, which are induced by presentation of simultaneous auditory stimuli, are the result of auditorily driven processing changes in visual motion processing areas. Through these studies, we will begin to explore the neural mechanisms underlying the process of "binding," related inputs across the separate sensory modalities.