Retrospectively supervised click decoder calibration for self-calibrating point-and-click brain–computer interfaces

Beata Jarosiewicz, Anish A. Sarma, Jad Saab, Brian Franco, Sydney S. Cash, Emad N. Eskandar, Leigh R. Hochberg

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Brain-computer interfaces (BCIs) aim to restore independence to people with severe motor disabilities by allowing control of a cursor on a computer screen or other effectors with neural activity. However, physiological and/or recording-related nonstationarities in neural signals can limit long-term decoding stability, and it would be tedious for users to pause use of the BCI whenever neural control degrades to perform decoder recalibration routines. We recently demonstrated that a kinematic decoder (i.e. a decoder that controls cursor movement) can be recalibrated using data acquired during practical point-and-click control of the BCI by retrospectively inferring users’ intended movement directions based on their subsequent selections. Here, we extend these methods to allow the click decoder to also be recalibrated using data acquired during practical BCI use. We retrospectively labeled neural data patterns as corresponding to “click” during all time bins in which the click log-likelihood (decoded using linear discriminant analysis, or LDA) had been above the click threshold that was used during real-time neural control. We labeled as “non-click” those periods that the kinematic decoder's retrospective target inference (RTI) heuristics determined to be consistent with intended cursor movement. Once these neural activity patterns were labeled, the click decoder was calibrated using standard supervised classifier training methods. Combined with real-time bias correction and baseline firing rate tracking, this set of “retrospectively labeled” decoder calibration methods enabled a BrainGate participant with amyotrophic lateral sclerosis (T9) to type freely across 11 research sessions spanning 29 days, maintaining high-performance neural control over cursor movement and click without needing to interrupt virtual keyboard use for explicit calibration tasks. By eliminating the need for tedious calibration tasks with prescribed targets and pre-specified click times, this approach advances the potential clinical utility of intracortical BCIs for individuals with severe motor disability.

Original languageEnglish (US)
Pages (from-to)382-391
Number of pages10
JournalJournal of Physiology Paris
Volume110
Issue number4
DOIs
StatePublished - Nov 1 2016
Externally publishedYes

Fingerprint

Brain-Computer Interfaces
Calibration
Biomechanical Phenomena
Amyotrophic Lateral Sclerosis
Discriminant Analysis
Research

Keywords

  • Adaptive classification
  • Amyotrophic lateral sclerosis (ALS)
  • Augmentative and assistive communication (AAC)
  • Brain-Machine Interface (BMI)
  • Spinal cord injury (SCI)
  • Stroke

ASJC Scopus subject areas

  • Neuroscience(all)
  • Physiology (medical)

Cite this

Retrospectively supervised click decoder calibration for self-calibrating point-and-click brain–computer interfaces. / Jarosiewicz, Beata; Sarma, Anish A.; Saab, Jad; Franco, Brian; Cash, Sydney S.; Eskandar, Emad N.; Hochberg, Leigh R.

In: Journal of Physiology Paris, Vol. 110, No. 4, 01.11.2016, p. 382-391.

Research output: Contribution to journalArticle

Jarosiewicz, Beata ; Sarma, Anish A. ; Saab, Jad ; Franco, Brian ; Cash, Sydney S. ; Eskandar, Emad N. ; Hochberg, Leigh R. / Retrospectively supervised click decoder calibration for self-calibrating point-and-click brain–computer interfaces. In: Journal of Physiology Paris. 2016 ; Vol. 110, No. 4. pp. 382-391.
@article{1681815f17d14e1baa5276e183c9dd2a,
title = "Retrospectively supervised click decoder calibration for self-calibrating point-and-click brain–computer interfaces",
abstract = "Brain-computer interfaces (BCIs) aim to restore independence to people with severe motor disabilities by allowing control of a cursor on a computer screen or other effectors with neural activity. However, physiological and/or recording-related nonstationarities in neural signals can limit long-term decoding stability, and it would be tedious for users to pause use of the BCI whenever neural control degrades to perform decoder recalibration routines. We recently demonstrated that a kinematic decoder (i.e. a decoder that controls cursor movement) can be recalibrated using data acquired during practical point-and-click control of the BCI by retrospectively inferring users’ intended movement directions based on their subsequent selections. Here, we extend these methods to allow the click decoder to also be recalibrated using data acquired during practical BCI use. We retrospectively labeled neural data patterns as corresponding to “click” during all time bins in which the click log-likelihood (decoded using linear discriminant analysis, or LDA) had been above the click threshold that was used during real-time neural control. We labeled as “non-click” those periods that the kinematic decoder's retrospective target inference (RTI) heuristics determined to be consistent with intended cursor movement. Once these neural activity patterns were labeled, the click decoder was calibrated using standard supervised classifier training methods. Combined with real-time bias correction and baseline firing rate tracking, this set of “retrospectively labeled” decoder calibration methods enabled a BrainGate participant with amyotrophic lateral sclerosis (T9) to type freely across 11 research sessions spanning 29 days, maintaining high-performance neural control over cursor movement and click without needing to interrupt virtual keyboard use for explicit calibration tasks. By eliminating the need for tedious calibration tasks with prescribed targets and pre-specified click times, this approach advances the potential clinical utility of intracortical BCIs for individuals with severe motor disability.",
keywords = "Adaptive classification, Amyotrophic lateral sclerosis (ALS), Augmentative and assistive communication (AAC), Brain-Machine Interface (BMI), Spinal cord injury (SCI), Stroke",
author = "Beata Jarosiewicz and Sarma, {Anish A.} and Jad Saab and Brian Franco and Cash, {Sydney S.} and Eskandar, {Emad N.} and Hochberg, {Leigh R.}",
year = "2016",
month = "11",
day = "1",
doi = "10.1016/j.jphysparis.2017.03.001",
language = "English (US)",
volume = "110",
pages = "382--391",
journal = "Journal de Physiologie",
issn = "0928-4257",
publisher = "Elsevier Masson SAS",
number = "4",

}

TY - JOUR

T1 - Retrospectively supervised click decoder calibration for self-calibrating point-and-click brain–computer interfaces

AU - Jarosiewicz, Beata

AU - Sarma, Anish A.

AU - Saab, Jad

AU - Franco, Brian

AU - Cash, Sydney S.

AU - Eskandar, Emad N.

AU - Hochberg, Leigh R.

PY - 2016/11/1

Y1 - 2016/11/1

N2 - Brain-computer interfaces (BCIs) aim to restore independence to people with severe motor disabilities by allowing control of a cursor on a computer screen or other effectors with neural activity. However, physiological and/or recording-related nonstationarities in neural signals can limit long-term decoding stability, and it would be tedious for users to pause use of the BCI whenever neural control degrades to perform decoder recalibration routines. We recently demonstrated that a kinematic decoder (i.e. a decoder that controls cursor movement) can be recalibrated using data acquired during practical point-and-click control of the BCI by retrospectively inferring users’ intended movement directions based on their subsequent selections. Here, we extend these methods to allow the click decoder to also be recalibrated using data acquired during practical BCI use. We retrospectively labeled neural data patterns as corresponding to “click” during all time bins in which the click log-likelihood (decoded using linear discriminant analysis, or LDA) had been above the click threshold that was used during real-time neural control. We labeled as “non-click” those periods that the kinematic decoder's retrospective target inference (RTI) heuristics determined to be consistent with intended cursor movement. Once these neural activity patterns were labeled, the click decoder was calibrated using standard supervised classifier training methods. Combined with real-time bias correction and baseline firing rate tracking, this set of “retrospectively labeled” decoder calibration methods enabled a BrainGate participant with amyotrophic lateral sclerosis (T9) to type freely across 11 research sessions spanning 29 days, maintaining high-performance neural control over cursor movement and click without needing to interrupt virtual keyboard use for explicit calibration tasks. By eliminating the need for tedious calibration tasks with prescribed targets and pre-specified click times, this approach advances the potential clinical utility of intracortical BCIs for individuals with severe motor disability.

AB - Brain-computer interfaces (BCIs) aim to restore independence to people with severe motor disabilities by allowing control of a cursor on a computer screen or other effectors with neural activity. However, physiological and/or recording-related nonstationarities in neural signals can limit long-term decoding stability, and it would be tedious for users to pause use of the BCI whenever neural control degrades to perform decoder recalibration routines. We recently demonstrated that a kinematic decoder (i.e. a decoder that controls cursor movement) can be recalibrated using data acquired during practical point-and-click control of the BCI by retrospectively inferring users’ intended movement directions based on their subsequent selections. Here, we extend these methods to allow the click decoder to also be recalibrated using data acquired during practical BCI use. We retrospectively labeled neural data patterns as corresponding to “click” during all time bins in which the click log-likelihood (decoded using linear discriminant analysis, or LDA) had been above the click threshold that was used during real-time neural control. We labeled as “non-click” those periods that the kinematic decoder's retrospective target inference (RTI) heuristics determined to be consistent with intended cursor movement. Once these neural activity patterns were labeled, the click decoder was calibrated using standard supervised classifier training methods. Combined with real-time bias correction and baseline firing rate tracking, this set of “retrospectively labeled” decoder calibration methods enabled a BrainGate participant with amyotrophic lateral sclerosis (T9) to type freely across 11 research sessions spanning 29 days, maintaining high-performance neural control over cursor movement and click without needing to interrupt virtual keyboard use for explicit calibration tasks. By eliminating the need for tedious calibration tasks with prescribed targets and pre-specified click times, this approach advances the potential clinical utility of intracortical BCIs for individuals with severe motor disability.

KW - Adaptive classification

KW - Amyotrophic lateral sclerosis (ALS)

KW - Augmentative and assistive communication (AAC)

KW - Brain-Machine Interface (BMI)

KW - Spinal cord injury (SCI)

KW - Stroke

UR - http://www.scopus.com/inward/record.url?scp=85015284815&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85015284815&partnerID=8YFLogxK

U2 - 10.1016/j.jphysparis.2017.03.001

DO - 10.1016/j.jphysparis.2017.03.001

M3 - Article

VL - 110

SP - 382

EP - 391

JO - Journal de Physiologie

JF - Journal de Physiologie

SN - 0928-4257

IS - 4

ER -