The correlation of workplace simulation-based assessments with interns' infant lumbar puncture success a prospective, multicenter, observational study

Marc Auerbach, Daniel M. Fein, Todd P. Chang, James Gerard, Pavan Zaveri, Devin Grossman, Wendy Van Ittersum, Joshua Rocker, Travis Whitfill, Martin Pusic, David O. Kessler

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Introduction: Little data are available to guide supervisors' decisions regarding when trainees are prepared to safely perform their first procedure on a patient. We aimed to describe the correlation of simulation-based assessments, in the workplace, with interns' first clinical infant lumbar puncture (ILP) success. Methods: This is a prospective, observational subcomponent of a larger study of incoming interns at 33 academic medical centers (July 2010 to June 2012) assessing the impact of just-in-time training.When an intern's patient required an ILP, a just-in-time simulationbased skills refresher was conducted with his or her supervisor. At the end of the refresher, supervisors assessed interns' ILP skills on a simulator in the workplace before clinical performance using a four point anchored scale. The primary outcome was the correlation of supervisors' assessment and interns' procedural success. The number needed to assess for this instrument (1/absolute risk reduction) was calculated. Results: A total of 1600 interns were eligible to participate, and 1215 were enrolled. A total of 297 completed an assessment and a subsequent clinical ILP. Success rates for each scale rating were 29% (18/63) for novice, 39% (51/130) for beginner, 55% (46/83) for competent, and 43% (9/21) for proficient. The correlation coefficient was 0.161 (95% confidence interval, 0.057Y0.265), indicating a weak correlation between supervisor rating and success. Success rate was 53% for the ratings of competent or proficient compared with 35% for the ratings of novice or beginner. Using the global rating scale for the summative assessment to determine procedural readiness could lead to 1 fewer patient experiencing a failed ILP for every 6 interns tested (6.2; 95% confidence interval, 4.0Y8.5). Conclusions: A simulation-based assessment of interns conducted in the workplace before their first ILP has some value in predicting clinical ILP success.

Original languageEnglish (US)
Pages (from-to)126-133
Number of pages8
JournalSimulation in Healthcare
Volume11
Issue number2
DOIs
StatePublished - 2016

Fingerprint

Observational Study
Spinal Puncture
Supervisory personnel
Workplace
Multicenter Studies
Observational Studies
infant
workplace
simulation
Simulation
Confidence interval
rating
rating scale
Personnel rating
confidence
Confidence Intervals
Correlation coefficient
Numbers Needed To Treat
Simulator
Simulators

Keywords

  • Global rating scale
  • Graduate medical education
  • Infant
  • Lumbar puncture
  • Pediatrics
  • Quality
  • Safety
  • Simulation
  • Workplace-based assessment

ASJC Scopus subject areas

  • Medicine (miscellaneous)
  • Epidemiology
  • Education
  • Modeling and Simulation

Cite this

The correlation of workplace simulation-based assessments with interns' infant lumbar puncture success a prospective, multicenter, observational study. / Auerbach, Marc; Fein, Daniel M.; Chang, Todd P.; Gerard, James; Zaveri, Pavan; Grossman, Devin; Van Ittersum, Wendy; Rocker, Joshua; Whitfill, Travis; Pusic, Martin; Kessler, David O.

In: Simulation in Healthcare, Vol. 11, No. 2, 2016, p. 126-133.

Research output: Contribution to journalArticle

Auerbach, M, Fein, DM, Chang, TP, Gerard, J, Zaveri, P, Grossman, D, Van Ittersum, W, Rocker, J, Whitfill, T, Pusic, M & Kessler, DO 2016, 'The correlation of workplace simulation-based assessments with interns' infant lumbar puncture success a prospective, multicenter, observational study', Simulation in Healthcare, vol. 11, no. 2, pp. 126-133. https://doi.org/10.1097/SIH.0000000000000135
Auerbach, Marc ; Fein, Daniel M. ; Chang, Todd P. ; Gerard, James ; Zaveri, Pavan ; Grossman, Devin ; Van Ittersum, Wendy ; Rocker, Joshua ; Whitfill, Travis ; Pusic, Martin ; Kessler, David O. / The correlation of workplace simulation-based assessments with interns' infant lumbar puncture success a prospective, multicenter, observational study. In: Simulation in Healthcare. 2016 ; Vol. 11, No. 2. pp. 126-133.
@article{c3c27bfc34334f338b9e0bb2637a83a0,
title = "The correlation of workplace simulation-based assessments with interns' infant lumbar puncture success a prospective, multicenter, observational study",
abstract = "Introduction: Little data are available to guide supervisors' decisions regarding when trainees are prepared to safely perform their first procedure on a patient. We aimed to describe the correlation of simulation-based assessments, in the workplace, with interns' first clinical infant lumbar puncture (ILP) success. Methods: This is a prospective, observational subcomponent of a larger study of incoming interns at 33 academic medical centers (July 2010 to June 2012) assessing the impact of just-in-time training.When an intern's patient required an ILP, a just-in-time simulationbased skills refresher was conducted with his or her supervisor. At the end of the refresher, supervisors assessed interns' ILP skills on a simulator in the workplace before clinical performance using a four point anchored scale. The primary outcome was the correlation of supervisors' assessment and interns' procedural success. The number needed to assess for this instrument (1/absolute risk reduction) was calculated. Results: A total of 1600 interns were eligible to participate, and 1215 were enrolled. A total of 297 completed an assessment and a subsequent clinical ILP. Success rates for each scale rating were 29{\%} (18/63) for novice, 39{\%} (51/130) for beginner, 55{\%} (46/83) for competent, and 43{\%} (9/21) for proficient. The correlation coefficient was 0.161 (95{\%} confidence interval, 0.057Y0.265), indicating a weak correlation between supervisor rating and success. Success rate was 53{\%} for the ratings of competent or proficient compared with 35{\%} for the ratings of novice or beginner. Using the global rating scale for the summative assessment to determine procedural readiness could lead to 1 fewer patient experiencing a failed ILP for every 6 interns tested (6.2; 95{\%} confidence interval, 4.0Y8.5). Conclusions: A simulation-based assessment of interns conducted in the workplace before their first ILP has some value in predicting clinical ILP success.",
keywords = "Global rating scale, Graduate medical education, Infant, Lumbar puncture, Pediatrics, Quality, Safety, Simulation, Workplace-based assessment",
author = "Marc Auerbach and Fein, {Daniel M.} and Chang, {Todd P.} and James Gerard and Pavan Zaveri and Devin Grossman and {Van Ittersum}, Wendy and Joshua Rocker and Travis Whitfill and Martin Pusic and Kessler, {David O.}",
year = "2016",
doi = "10.1097/SIH.0000000000000135",
language = "English (US)",
volume = "11",
pages = "126--133",
journal = "Simulation in Healthcare",
issn = "1559-2332",
publisher = "Lippincott Williams and Wilkins",
number = "2",

}

TY - JOUR

T1 - The correlation of workplace simulation-based assessments with interns' infant lumbar puncture success a prospective, multicenter, observational study

AU - Auerbach, Marc

AU - Fein, Daniel M.

AU - Chang, Todd P.

AU - Gerard, James

AU - Zaveri, Pavan

AU - Grossman, Devin

AU - Van Ittersum, Wendy

AU - Rocker, Joshua

AU - Whitfill, Travis

AU - Pusic, Martin

AU - Kessler, David O.

PY - 2016

Y1 - 2016

N2 - Introduction: Little data are available to guide supervisors' decisions regarding when trainees are prepared to safely perform their first procedure on a patient. We aimed to describe the correlation of simulation-based assessments, in the workplace, with interns' first clinical infant lumbar puncture (ILP) success. Methods: This is a prospective, observational subcomponent of a larger study of incoming interns at 33 academic medical centers (July 2010 to June 2012) assessing the impact of just-in-time training.When an intern's patient required an ILP, a just-in-time simulationbased skills refresher was conducted with his or her supervisor. At the end of the refresher, supervisors assessed interns' ILP skills on a simulator in the workplace before clinical performance using a four point anchored scale. The primary outcome was the correlation of supervisors' assessment and interns' procedural success. The number needed to assess for this instrument (1/absolute risk reduction) was calculated. Results: A total of 1600 interns were eligible to participate, and 1215 were enrolled. A total of 297 completed an assessment and a subsequent clinical ILP. Success rates for each scale rating were 29% (18/63) for novice, 39% (51/130) for beginner, 55% (46/83) for competent, and 43% (9/21) for proficient. The correlation coefficient was 0.161 (95% confidence interval, 0.057Y0.265), indicating a weak correlation between supervisor rating and success. Success rate was 53% for the ratings of competent or proficient compared with 35% for the ratings of novice or beginner. Using the global rating scale for the summative assessment to determine procedural readiness could lead to 1 fewer patient experiencing a failed ILP for every 6 interns tested (6.2; 95% confidence interval, 4.0Y8.5). Conclusions: A simulation-based assessment of interns conducted in the workplace before their first ILP has some value in predicting clinical ILP success.

AB - Introduction: Little data are available to guide supervisors' decisions regarding when trainees are prepared to safely perform their first procedure on a patient. We aimed to describe the correlation of simulation-based assessments, in the workplace, with interns' first clinical infant lumbar puncture (ILP) success. Methods: This is a prospective, observational subcomponent of a larger study of incoming interns at 33 academic medical centers (July 2010 to June 2012) assessing the impact of just-in-time training.When an intern's patient required an ILP, a just-in-time simulationbased skills refresher was conducted with his or her supervisor. At the end of the refresher, supervisors assessed interns' ILP skills on a simulator in the workplace before clinical performance using a four point anchored scale. The primary outcome was the correlation of supervisors' assessment and interns' procedural success. The number needed to assess for this instrument (1/absolute risk reduction) was calculated. Results: A total of 1600 interns were eligible to participate, and 1215 were enrolled. A total of 297 completed an assessment and a subsequent clinical ILP. Success rates for each scale rating were 29% (18/63) for novice, 39% (51/130) for beginner, 55% (46/83) for competent, and 43% (9/21) for proficient. The correlation coefficient was 0.161 (95% confidence interval, 0.057Y0.265), indicating a weak correlation between supervisor rating and success. Success rate was 53% for the ratings of competent or proficient compared with 35% for the ratings of novice or beginner. Using the global rating scale for the summative assessment to determine procedural readiness could lead to 1 fewer patient experiencing a failed ILP for every 6 interns tested (6.2; 95% confidence interval, 4.0Y8.5). Conclusions: A simulation-based assessment of interns conducted in the workplace before their first ILP has some value in predicting clinical ILP success.

KW - Global rating scale

KW - Graduate medical education

KW - Infant

KW - Lumbar puncture

KW - Pediatrics

KW - Quality

KW - Safety

KW - Simulation

KW - Workplace-based assessment

UR - http://www.scopus.com/inward/record.url?scp=84964055241&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84964055241&partnerID=8YFLogxK

U2 - 10.1097/SIH.0000000000000135

DO - 10.1097/SIH.0000000000000135

M3 - Article

C2 - 27043098

AN - SCOPUS:84964055241

VL - 11

SP - 126

EP - 133

JO - Simulation in Healthcare

JF - Simulation in Healthcare

SN - 1559-2332

IS - 2

ER -