Interrater reliability among epilepsy centers

Multicenter study of epilepsy surgery

Sheryl R. Haut, Anne T. Berg, Shlomo Shinnar, Hillel W. Cohen, Carl W. Bazil, Michael R. Sperling, John T. Langfitt, Steven V. Pacia, Thaddeus S. Walczak, Susan S. Spencer

Research output: Contribution to journalArticle

27 Citations (Scopus)

Abstract

Purpose: To measure the interrater reliability of presurgical testing and surgical decisions among epilepsy centers. Methods: Seven centers participating in an ongoing, prospective multicenter study of resective epilepsy surgery agreed to conform to a detailed protocol regarding presurgical evaluation and surgery. To assess quality assurance, each center independently reviewed 21 randomly selected surgical cases for preoperative study lateralization and localization, and surgical decisions. Interrater reliability was assessed by using intraclass correlation coefficients (ICCs), validated for use with multiple raters, and calculated in a two-way random model based on absolute agreement. Results: Agreement for ICC values: ≥0.75, excellent; 0.60-0.74, good; 0.40-0.59, fair; ≤0.39, poor. One center was excluded for missing data. Agreement was excellent for extracranial EEG lateralization (0.8039), magnetic resonance imaging (MRI) lateralization (0.9521) and localization (0.9130), Wada lateralization (0.9453), and intracranial EEG localization (0.7905). Agreement was good for extracranial EEG localization (0.7384) and neuropsychological testing lateralization (0,7178) and localization (0.6891). Consensus about the decision to perform intracranial monitoring was fair (0.5397), in part reflecting one center's tendency toward intracranial monitoring. Overall agreements on whether to perform surgery (0.8311) and specific surgery recommended (0.8164) were excellent. Conclusions: High interrater reliability among six epilepsy centers was present for interpretation of most components of presurgical testing. Although consensus for the decision to perform intracranial monitoring was only fair, agreements for the ultimate decision about resective surgery and specific choice of resection were excellent. We believe that this study demonstrates the feasibility of implementing multicenter protocols for neurologic management, especially those involving localization, as well as protocols combining study results with clinical decision making.

Original languageEnglish (US)
Pages (from-to)1396-1401
Number of pages6
JournalEpilepsia
Volume43
Issue number11
DOIs
StatePublished - Nov 1 2002

Fingerprint

Multicenter Studies
Epilepsy
Electroencephalography
Feasibility Studies
Nervous System
Magnetic Resonance Imaging
Prospective Studies

Keywords

  • Interrater reliability
  • Intraclass correlation coefficients
  • Surgery

ASJC Scopus subject areas

  • Clinical Neurology
  • Neuroscience(all)

Cite this

Haut, S. R., Berg, A. T., Shinnar, S., Cohen, H. W., Bazil, C. W., Sperling, M. R., ... Spencer, S. S. (2002). Interrater reliability among epilepsy centers: Multicenter study of epilepsy surgery. Epilepsia, 43(11), 1396-1401. https://doi.org/10.1046/j.1528-1157.2002.20902.x

Interrater reliability among epilepsy centers : Multicenter study of epilepsy surgery. / Haut, Sheryl R.; Berg, Anne T.; Shinnar, Shlomo; Cohen, Hillel W.; Bazil, Carl W.; Sperling, Michael R.; Langfitt, John T.; Pacia, Steven V.; Walczak, Thaddeus S.; Spencer, Susan S.

In: Epilepsia, Vol. 43, No. 11, 01.11.2002, p. 1396-1401.

Research output: Contribution to journalArticle

Haut, SR, Berg, AT, Shinnar, S, Cohen, HW, Bazil, CW, Sperling, MR, Langfitt, JT, Pacia, SV, Walczak, TS & Spencer, SS 2002, 'Interrater reliability among epilepsy centers: Multicenter study of epilepsy surgery', Epilepsia, vol. 43, no. 11, pp. 1396-1401. https://doi.org/10.1046/j.1528-1157.2002.20902.x
Haut, Sheryl R. ; Berg, Anne T. ; Shinnar, Shlomo ; Cohen, Hillel W. ; Bazil, Carl W. ; Sperling, Michael R. ; Langfitt, John T. ; Pacia, Steven V. ; Walczak, Thaddeus S. ; Spencer, Susan S. / Interrater reliability among epilepsy centers : Multicenter study of epilepsy surgery. In: Epilepsia. 2002 ; Vol. 43, No. 11. pp. 1396-1401.
@article{43d382a8823948ac99b3f30103ec9679,
title = "Interrater reliability among epilepsy centers: Multicenter study of epilepsy surgery",
abstract = "Purpose: To measure the interrater reliability of presurgical testing and surgical decisions among epilepsy centers. Methods: Seven centers participating in an ongoing, prospective multicenter study of resective epilepsy surgery agreed to conform to a detailed protocol regarding presurgical evaluation and surgery. To assess quality assurance, each center independently reviewed 21 randomly selected surgical cases for preoperative study lateralization and localization, and surgical decisions. Interrater reliability was assessed by using intraclass correlation coefficients (ICCs), validated for use with multiple raters, and calculated in a two-way random model based on absolute agreement. Results: Agreement for ICC values: ≥0.75, excellent; 0.60-0.74, good; 0.40-0.59, fair; ≤0.39, poor. One center was excluded for missing data. Agreement was excellent for extracranial EEG lateralization (0.8039), magnetic resonance imaging (MRI) lateralization (0.9521) and localization (0.9130), Wada lateralization (0.9453), and intracranial EEG localization (0.7905). Agreement was good for extracranial EEG localization (0.7384) and neuropsychological testing lateralization (0,7178) and localization (0.6891). Consensus about the decision to perform intracranial monitoring was fair (0.5397), in part reflecting one center's tendency toward intracranial monitoring. Overall agreements on whether to perform surgery (0.8311) and specific surgery recommended (0.8164) were excellent. Conclusions: High interrater reliability among six epilepsy centers was present for interpretation of most components of presurgical testing. Although consensus for the decision to perform intracranial monitoring was only fair, agreements for the ultimate decision about resective surgery and specific choice of resection were excellent. We believe that this study demonstrates the feasibility of implementing multicenter protocols for neurologic management, especially those involving localization, as well as protocols combining study results with clinical decision making.",
keywords = "Interrater reliability, Intraclass correlation coefficients, Surgery",
author = "Haut, {Sheryl R.} and Berg, {Anne T.} and Shlomo Shinnar and Cohen, {Hillel W.} and Bazil, {Carl W.} and Sperling, {Michael R.} and Langfitt, {John T.} and Pacia, {Steven V.} and Walczak, {Thaddeus S.} and Spencer, {Susan S.}",
year = "2002",
month = "11",
day = "1",
doi = "10.1046/j.1528-1157.2002.20902.x",
language = "English (US)",
volume = "43",
pages = "1396--1401",
journal = "Epilepsia",
issn = "0013-9580",
publisher = "Wiley-Blackwell",
number = "11",

}

TY - JOUR

T1 - Interrater reliability among epilepsy centers

T2 - Multicenter study of epilepsy surgery

AU - Haut, Sheryl R.

AU - Berg, Anne T.

AU - Shinnar, Shlomo

AU - Cohen, Hillel W.

AU - Bazil, Carl W.

AU - Sperling, Michael R.

AU - Langfitt, John T.

AU - Pacia, Steven V.

AU - Walczak, Thaddeus S.

AU - Spencer, Susan S.

PY - 2002/11/1

Y1 - 2002/11/1

N2 - Purpose: To measure the interrater reliability of presurgical testing and surgical decisions among epilepsy centers. Methods: Seven centers participating in an ongoing, prospective multicenter study of resective epilepsy surgery agreed to conform to a detailed protocol regarding presurgical evaluation and surgery. To assess quality assurance, each center independently reviewed 21 randomly selected surgical cases for preoperative study lateralization and localization, and surgical decisions. Interrater reliability was assessed by using intraclass correlation coefficients (ICCs), validated for use with multiple raters, and calculated in a two-way random model based on absolute agreement. Results: Agreement for ICC values: ≥0.75, excellent; 0.60-0.74, good; 0.40-0.59, fair; ≤0.39, poor. One center was excluded for missing data. Agreement was excellent for extracranial EEG lateralization (0.8039), magnetic resonance imaging (MRI) lateralization (0.9521) and localization (0.9130), Wada lateralization (0.9453), and intracranial EEG localization (0.7905). Agreement was good for extracranial EEG localization (0.7384) and neuropsychological testing lateralization (0,7178) and localization (0.6891). Consensus about the decision to perform intracranial monitoring was fair (0.5397), in part reflecting one center's tendency toward intracranial monitoring. Overall agreements on whether to perform surgery (0.8311) and specific surgery recommended (0.8164) were excellent. Conclusions: High interrater reliability among six epilepsy centers was present for interpretation of most components of presurgical testing. Although consensus for the decision to perform intracranial monitoring was only fair, agreements for the ultimate decision about resective surgery and specific choice of resection were excellent. We believe that this study demonstrates the feasibility of implementing multicenter protocols for neurologic management, especially those involving localization, as well as protocols combining study results with clinical decision making.

AB - Purpose: To measure the interrater reliability of presurgical testing and surgical decisions among epilepsy centers. Methods: Seven centers participating in an ongoing, prospective multicenter study of resective epilepsy surgery agreed to conform to a detailed protocol regarding presurgical evaluation and surgery. To assess quality assurance, each center independently reviewed 21 randomly selected surgical cases for preoperative study lateralization and localization, and surgical decisions. Interrater reliability was assessed by using intraclass correlation coefficients (ICCs), validated for use with multiple raters, and calculated in a two-way random model based on absolute agreement. Results: Agreement for ICC values: ≥0.75, excellent; 0.60-0.74, good; 0.40-0.59, fair; ≤0.39, poor. One center was excluded for missing data. Agreement was excellent for extracranial EEG lateralization (0.8039), magnetic resonance imaging (MRI) lateralization (0.9521) and localization (0.9130), Wada lateralization (0.9453), and intracranial EEG localization (0.7905). Agreement was good for extracranial EEG localization (0.7384) and neuropsychological testing lateralization (0,7178) and localization (0.6891). Consensus about the decision to perform intracranial monitoring was fair (0.5397), in part reflecting one center's tendency toward intracranial monitoring. Overall agreements on whether to perform surgery (0.8311) and specific surgery recommended (0.8164) were excellent. Conclusions: High interrater reliability among six epilepsy centers was present for interpretation of most components of presurgical testing. Although consensus for the decision to perform intracranial monitoring was only fair, agreements for the ultimate decision about resective surgery and specific choice of resection were excellent. We believe that this study demonstrates the feasibility of implementing multicenter protocols for neurologic management, especially those involving localization, as well as protocols combining study results with clinical decision making.

KW - Interrater reliability

KW - Intraclass correlation coefficients

KW - Surgery

UR - http://www.scopus.com/inward/record.url?scp=18744391062&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=18744391062&partnerID=8YFLogxK

U2 - 10.1046/j.1528-1157.2002.20902.x

DO - 10.1046/j.1528-1157.2002.20902.x

M3 - Article

VL - 43

SP - 1396

EP - 1401

JO - Epilepsia

JF - Epilepsia

SN - 0013-9580

IS - 11

ER -