The correlation of workplace simulation-based assessments with interns' infant lumbar puncture success a prospective, multicenter, observational study

Marc Auerbach, Daniel M. Fein, Todd P. Chang, James Gerard, Pavan Zaveri, Devin Grossman, Wendy Van Ittersum, Joshua Rocker, Travis Whitfill, Martin Pusic, David O. Kessler

Research output: Contribution to journalArticle

4 Scopus citations


Introduction: Little data are available to guide supervisors' decisions regarding when trainees are prepared to safely perform their first procedure on a patient. We aimed to describe the correlation of simulation-based assessments, in the workplace, with interns' first clinical infant lumbar puncture (ILP) success. Methods: This is a prospective, observational subcomponent of a larger study of incoming interns at 33 academic medical centers (July 2010 to June 2012) assessing the impact of just-in-time training.When an intern's patient required an ILP, a just-in-time simulationbased skills refresher was conducted with his or her supervisor. At the end of the refresher, supervisors assessed interns' ILP skills on a simulator in the workplace before clinical performance using a four point anchored scale. The primary outcome was the correlation of supervisors' assessment and interns' procedural success. The number needed to assess for this instrument (1/absolute risk reduction) was calculated. Results: A total of 1600 interns were eligible to participate, and 1215 were enrolled. A total of 297 completed an assessment and a subsequent clinical ILP. Success rates for each scale rating were 29% (18/63) for novice, 39% (51/130) for beginner, 55% (46/83) for competent, and 43% (9/21) for proficient. The correlation coefficient was 0.161 (95% confidence interval, 0.057Y0.265), indicating a weak correlation between supervisor rating and success. Success rate was 53% for the ratings of competent or proficient compared with 35% for the ratings of novice or beginner. Using the global rating scale for the summative assessment to determine procedural readiness could lead to 1 fewer patient experiencing a failed ILP for every 6 interns tested (6.2; 95% confidence interval, 4.0Y8.5). Conclusions: A simulation-based assessment of interns conducted in the workplace before their first ILP has some value in predicting clinical ILP success.

Original languageEnglish (US)
Pages (from-to)126-133
Number of pages8
JournalSimulation in Healthcare
Issue number2
StatePublished - Jan 1 2016



  • Global rating scale
  • Graduate medical education
  • Infant
  • Lumbar puncture
  • Pediatrics
  • Quality
  • Safety
  • Simulation
  • Workplace-based assessment

ASJC Scopus subject areas

  • Epidemiology
  • Medicine (miscellaneous)
  • Education
  • Modeling and Simulation

Cite this

Auerbach, M., Fein, D. M., Chang, T. P., Gerard, J., Zaveri, P., Grossman, D., Van Ittersum, W., Rocker, J., Whitfill, T., Pusic, M., & Kessler, D. O. (2016). The correlation of workplace simulation-based assessments with interns' infant lumbar puncture success a prospective, multicenter, observational study. Simulation in Healthcare, 11(2), 126-133.