Developing end-of-training entrustable professional activities for psychiatry: Results and methodological lessons

John Q. Young, Caitlin Hasser, Erick K. Hung, Martin Kusz, Patricia S. O'Sullivan, Colin Stewart, Andrea J. Weiss, Nancy Williams

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

Purpose To develop entrustable professional activities (EPAs) for psychiatry and to demonstrate an innovative, validity-enhancing methodology that may be relevant to other specialties. Method A national task force employed a three-stage process from May 2014 to February 2017 to develop EPAs for psychiatry. In stage 1, the task force used an iterative consensus-driven process to construct proposed EPAs. Each included a title, full description, and relevant competencies. In stage 2, the task force interviewed four nonpsychiatric experts in EPAs and further revised the EPAs. In stage 3, the task force performed a Delphi study of national experts in psychiatric education and assessment. All survey participants completed a brief training program on EPAs. Quantitative and qualitative analysis led to further modifications. Essentialness was measured on a five-point scale. EPAs were included if the content validity index was at least 0.8 and the lower end of the asymmetric confidence interval was not lower than 4.0. Results Stages 1 and 2 yielded 24 and 14 EPAs, respectively. In stage 3, 31 of the 39 invited experts participated in both rounds of the Delphi study. Round 1 reduced the proposed EPAs to 13. Ten EPAs met the inclusion criteria in Round 2. Conclusions The final EPAs provide a strong foundation for competency-based assessment in psychiatry. Methodological features such as critique by nonpsychiatry experts, a national Delphi study with frame-of-reference training, and stringent inclusion criteria strengthen the content validity of the findings and may serve as a model for future efforts in other specialties.

Original languageEnglish (US)
Pages (from-to)1048-1054
Number of pages7
JournalAcademic Medicine
Volume93
Issue number7
DOIs
StatePublished - Jul 1 2018

Fingerprint

psychiatry
expert
inclusion
training program
confidence
methodology

ASJC Scopus subject areas

  • Education

Cite this

Young, J. Q., Hasser, C., Hung, E. K., Kusz, M., O'Sullivan, P. S., Stewart, C., ... Williams, N. (2018). Developing end-of-training entrustable professional activities for psychiatry: Results and methodological lessons. Academic Medicine, 93(7), 1048-1054. https://doi.org/10.1097/ACM.0000000000002058

Developing end-of-training entrustable professional activities for psychiatry : Results and methodological lessons. / Young, John Q.; Hasser, Caitlin; Hung, Erick K.; Kusz, Martin; O'Sullivan, Patricia S.; Stewart, Colin; Weiss, Andrea J.; Williams, Nancy.

In: Academic Medicine, Vol. 93, No. 7, 01.07.2018, p. 1048-1054.

Research output: Contribution to journalArticle

Young, JQ, Hasser, C, Hung, EK, Kusz, M, O'Sullivan, PS, Stewart, C, Weiss, AJ & Williams, N 2018, 'Developing end-of-training entrustable professional activities for psychiatry: Results and methodological lessons', Academic Medicine, vol. 93, no. 7, pp. 1048-1054. https://doi.org/10.1097/ACM.0000000000002058
Young, John Q. ; Hasser, Caitlin ; Hung, Erick K. ; Kusz, Martin ; O'Sullivan, Patricia S. ; Stewart, Colin ; Weiss, Andrea J. ; Williams, Nancy. / Developing end-of-training entrustable professional activities for psychiatry : Results and methodological lessons. In: Academic Medicine. 2018 ; Vol. 93, No. 7. pp. 1048-1054.
@article{6df040f7fb1441c483c128005243ac1a,
title = "Developing end-of-training entrustable professional activities for psychiatry: Results and methodological lessons",
abstract = "Purpose To develop entrustable professional activities (EPAs) for psychiatry and to demonstrate an innovative, validity-enhancing methodology that may be relevant to other specialties. Method A national task force employed a three-stage process from May 2014 to February 2017 to develop EPAs for psychiatry. In stage 1, the task force used an iterative consensus-driven process to construct proposed EPAs. Each included a title, full description, and relevant competencies. In stage 2, the task force interviewed four nonpsychiatric experts in EPAs and further revised the EPAs. In stage 3, the task force performed a Delphi study of national experts in psychiatric education and assessment. All survey participants completed a brief training program on EPAs. Quantitative and qualitative analysis led to further modifications. Essentialness was measured on a five-point scale. EPAs were included if the content validity index was at least 0.8 and the lower end of the asymmetric confidence interval was not lower than 4.0. Results Stages 1 and 2 yielded 24 and 14 EPAs, respectively. In stage 3, 31 of the 39 invited experts participated in both rounds of the Delphi study. Round 1 reduced the proposed EPAs to 13. Ten EPAs met the inclusion criteria in Round 2. Conclusions The final EPAs provide a strong foundation for competency-based assessment in psychiatry. Methodological features such as critique by nonpsychiatry experts, a national Delphi study with frame-of-reference training, and stringent inclusion criteria strengthen the content validity of the findings and may serve as a model for future efforts in other specialties.",
author = "Young, {John Q.} and Caitlin Hasser and Hung, {Erick K.} and Martin Kusz and O'Sullivan, {Patricia S.} and Colin Stewart and Weiss, {Andrea J.} and Nancy Williams",
year = "2018",
month = "7",
day = "1",
doi = "10.1097/ACM.0000000000002058",
language = "English (US)",
volume = "93",
pages = "1048--1054",
journal = "Academic Medicine",
issn = "1040-2446",
publisher = "Lippincott Williams and Wilkins",
number = "7",

}

TY - JOUR

T1 - Developing end-of-training entrustable professional activities for psychiatry

T2 - Results and methodological lessons

AU - Young, John Q.

AU - Hasser, Caitlin

AU - Hung, Erick K.

AU - Kusz, Martin

AU - O'Sullivan, Patricia S.

AU - Stewart, Colin

AU - Weiss, Andrea J.

AU - Williams, Nancy

PY - 2018/7/1

Y1 - 2018/7/1

N2 - Purpose To develop entrustable professional activities (EPAs) for psychiatry and to demonstrate an innovative, validity-enhancing methodology that may be relevant to other specialties. Method A national task force employed a three-stage process from May 2014 to February 2017 to develop EPAs for psychiatry. In stage 1, the task force used an iterative consensus-driven process to construct proposed EPAs. Each included a title, full description, and relevant competencies. In stage 2, the task force interviewed four nonpsychiatric experts in EPAs and further revised the EPAs. In stage 3, the task force performed a Delphi study of national experts in psychiatric education and assessment. All survey participants completed a brief training program on EPAs. Quantitative and qualitative analysis led to further modifications. Essentialness was measured on a five-point scale. EPAs were included if the content validity index was at least 0.8 and the lower end of the asymmetric confidence interval was not lower than 4.0. Results Stages 1 and 2 yielded 24 and 14 EPAs, respectively. In stage 3, 31 of the 39 invited experts participated in both rounds of the Delphi study. Round 1 reduced the proposed EPAs to 13. Ten EPAs met the inclusion criteria in Round 2. Conclusions The final EPAs provide a strong foundation for competency-based assessment in psychiatry. Methodological features such as critique by nonpsychiatry experts, a national Delphi study with frame-of-reference training, and stringent inclusion criteria strengthen the content validity of the findings and may serve as a model for future efforts in other specialties.

AB - Purpose To develop entrustable professional activities (EPAs) for psychiatry and to demonstrate an innovative, validity-enhancing methodology that may be relevant to other specialties. Method A national task force employed a three-stage process from May 2014 to February 2017 to develop EPAs for psychiatry. In stage 1, the task force used an iterative consensus-driven process to construct proposed EPAs. Each included a title, full description, and relevant competencies. In stage 2, the task force interviewed four nonpsychiatric experts in EPAs and further revised the EPAs. In stage 3, the task force performed a Delphi study of national experts in psychiatric education and assessment. All survey participants completed a brief training program on EPAs. Quantitative and qualitative analysis led to further modifications. Essentialness was measured on a five-point scale. EPAs were included if the content validity index was at least 0.8 and the lower end of the asymmetric confidence interval was not lower than 4.0. Results Stages 1 and 2 yielded 24 and 14 EPAs, respectively. In stage 3, 31 of the 39 invited experts participated in both rounds of the Delphi study. Round 1 reduced the proposed EPAs to 13. Ten EPAs met the inclusion criteria in Round 2. Conclusions The final EPAs provide a strong foundation for competency-based assessment in psychiatry. Methodological features such as critique by nonpsychiatry experts, a national Delphi study with frame-of-reference training, and stringent inclusion criteria strengthen the content validity of the findings and may serve as a model for future efforts in other specialties.

UR - http://www.scopus.com/inward/record.url?scp=85057484786&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85057484786&partnerID=8YFLogxK

U2 - 10.1097/ACM.0000000000002058

DO - 10.1097/ACM.0000000000002058

M3 - Article

AN - SCOPUS:85057484786

VL - 93

SP - 1048

EP - 1054

JO - Academic Medicine

JF - Academic Medicine

SN - 1040-2446

IS - 7

ER -