Investigating the temporal dynamics of auditory cortical activation to silent lipreading

Michael J. Crosse, Hesham A. Elshafei, John J. Foxe, Edmund C. Lalor

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Neuroimaging research has demonstrated that observing visual speech in the absence of auditory speech activates primary auditory cortex. However, it remains unclear what this activation precisely reflects. It is well established that, during continuous auditory speech, neural activity in auditory cortex tracks the temporal envelope of the speech signal. Recently, it has been suggested that this process may in fact reflect an internal synthesis of the speech stream rather than the encoding of the envelope per se. Could silent lipreading therefore elicit a similar 'entrainment' to the envelope in the absence of auditory speech? Here, we test this hypothesis by examining the impact of lipreading accuracy on envelope tracking using electroencephalography (EEG). We provide evidence to suggest that the EEG response over left temporal scalp tracks the unheard speech more faithfully during accurate lipreading. We also demonstrate that the envelope can be reconstructed from EEG data recorded during silent lipreading with accuracy above chance level. This could have implications for brain-computer interface technology.

Original languageEnglish (US)
Title of host publicationInternational IEEE/EMBS Conference on Neural Engineering, NER
PublisherIEEE Computer Society
Pages308-311
Number of pages4
Volume2015-July
ISBN (Print)9781467363891
DOIs
StatePublished - Jul 1 2015
Event7th International IEEE/EMBS Conference on Neural Engineering, NER 2015 - Montpellier, France
Duration: Apr 22 2015Apr 24 2015

Other

Other7th International IEEE/EMBS Conference on Neural Engineering, NER 2015
CountryFrance
CityMontpellier
Period4/22/154/24/15

Fingerprint

Chemical activation
Electroencephalography
Neuroimaging
Brain computer interface

ASJC Scopus subject areas

  • Artificial Intelligence
  • Mechanical Engineering

Cite this

Crosse, M. J., Elshafei, H. A., Foxe, J. J., & Lalor, E. C. (2015). Investigating the temporal dynamics of auditory cortical activation to silent lipreading. In International IEEE/EMBS Conference on Neural Engineering, NER (Vol. 2015-July, pp. 308-311). [7146621] IEEE Computer Society. https://doi.org/10.1109/NER.2015.7146621

Investigating the temporal dynamics of auditory cortical activation to silent lipreading. / Crosse, Michael J.; Elshafei, Hesham A.; Foxe, John J.; Lalor, Edmund C.

International IEEE/EMBS Conference on Neural Engineering, NER. Vol. 2015-July IEEE Computer Society, 2015. p. 308-311 7146621.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Crosse, MJ, Elshafei, HA, Foxe, JJ & Lalor, EC 2015, Investigating the temporal dynamics of auditory cortical activation to silent lipreading. in International IEEE/EMBS Conference on Neural Engineering, NER. vol. 2015-July, 7146621, IEEE Computer Society, pp. 308-311, 7th International IEEE/EMBS Conference on Neural Engineering, NER 2015, Montpellier, France, 4/22/15. https://doi.org/10.1109/NER.2015.7146621
Crosse MJ, Elshafei HA, Foxe JJ, Lalor EC. Investigating the temporal dynamics of auditory cortical activation to silent lipreading. In International IEEE/EMBS Conference on Neural Engineering, NER. Vol. 2015-July. IEEE Computer Society. 2015. p. 308-311. 7146621 https://doi.org/10.1109/NER.2015.7146621
Crosse, Michael J. ; Elshafei, Hesham A. ; Foxe, John J. ; Lalor, Edmund C. / Investigating the temporal dynamics of auditory cortical activation to silent lipreading. International IEEE/EMBS Conference on Neural Engineering, NER. Vol. 2015-July IEEE Computer Society, 2015. pp. 308-311
@inproceedings{f912e9535c6a4171b05dca2fc0bffff2,
title = "Investigating the temporal dynamics of auditory cortical activation to silent lipreading",
abstract = "Neuroimaging research has demonstrated that observing visual speech in the absence of auditory speech activates primary auditory cortex. However, it remains unclear what this activation precisely reflects. It is well established that, during continuous auditory speech, neural activity in auditory cortex tracks the temporal envelope of the speech signal. Recently, it has been suggested that this process may in fact reflect an internal synthesis of the speech stream rather than the encoding of the envelope per se. Could silent lipreading therefore elicit a similar 'entrainment' to the envelope in the absence of auditory speech? Here, we test this hypothesis by examining the impact of lipreading accuracy on envelope tracking using electroencephalography (EEG). We provide evidence to suggest that the EEG response over left temporal scalp tracks the unheard speech more faithfully during accurate lipreading. We also demonstrate that the envelope can be reconstructed from EEG data recorded during silent lipreading with accuracy above chance level. This could have implications for brain-computer interface technology.",
author = "Crosse, {Michael J.} and Elshafei, {Hesham A.} and Foxe, {John J.} and Lalor, {Edmund C.}",
year = "2015",
month = "7",
day = "1",
doi = "10.1109/NER.2015.7146621",
language = "English (US)",
isbn = "9781467363891",
volume = "2015-July",
pages = "308--311",
booktitle = "International IEEE/EMBS Conference on Neural Engineering, NER",
publisher = "IEEE Computer Society",

}

TY - GEN

T1 - Investigating the temporal dynamics of auditory cortical activation to silent lipreading

AU - Crosse, Michael J.

AU - Elshafei, Hesham A.

AU - Foxe, John J.

AU - Lalor, Edmund C.

PY - 2015/7/1

Y1 - 2015/7/1

N2 - Neuroimaging research has demonstrated that observing visual speech in the absence of auditory speech activates primary auditory cortex. However, it remains unclear what this activation precisely reflects. It is well established that, during continuous auditory speech, neural activity in auditory cortex tracks the temporal envelope of the speech signal. Recently, it has been suggested that this process may in fact reflect an internal synthesis of the speech stream rather than the encoding of the envelope per se. Could silent lipreading therefore elicit a similar 'entrainment' to the envelope in the absence of auditory speech? Here, we test this hypothesis by examining the impact of lipreading accuracy on envelope tracking using electroencephalography (EEG). We provide evidence to suggest that the EEG response over left temporal scalp tracks the unheard speech more faithfully during accurate lipreading. We also demonstrate that the envelope can be reconstructed from EEG data recorded during silent lipreading with accuracy above chance level. This could have implications for brain-computer interface technology.

AB - Neuroimaging research has demonstrated that observing visual speech in the absence of auditory speech activates primary auditory cortex. However, it remains unclear what this activation precisely reflects. It is well established that, during continuous auditory speech, neural activity in auditory cortex tracks the temporal envelope of the speech signal. Recently, it has been suggested that this process may in fact reflect an internal synthesis of the speech stream rather than the encoding of the envelope per se. Could silent lipreading therefore elicit a similar 'entrainment' to the envelope in the absence of auditory speech? Here, we test this hypothesis by examining the impact of lipreading accuracy on envelope tracking using electroencephalography (EEG). We provide evidence to suggest that the EEG response over left temporal scalp tracks the unheard speech more faithfully during accurate lipreading. We also demonstrate that the envelope can be reconstructed from EEG data recorded during silent lipreading with accuracy above chance level. This could have implications for brain-computer interface technology.

UR - http://www.scopus.com/inward/record.url?scp=84940386154&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84940386154&partnerID=8YFLogxK

U2 - 10.1109/NER.2015.7146621

DO - 10.1109/NER.2015.7146621

M3 - Conference contribution

AN - SCOPUS:84940386154

SN - 9781467363891

VL - 2015-July

SP - 308

EP - 311

BT - International IEEE/EMBS Conference on Neural Engineering, NER

PB - IEEE Computer Society

ER -