Assessing the quality of clinical teaching: A preliminary study: Teacher assessment

Rosemarie L. Conigliaro, Terry D. Stratton

Research output: Contribution to journalArticle

15 Citations (Scopus)

Abstract

Objectives Evaluations in the clinical arena are fraught with problems. Current assessments of clinical teaching typically measure attributes of clinical teachers in overly broad terms, are often subjective and often succumb to the halo effect. This is in contradistinction to measurements of lectures, workshops or online educational content, which can more readily be assessed using objective criteria. As a result, clinical evaluations are often insufficient to provide focused feedback, guide faculty development or identify specific areas for clinical teachers to implement change and improvement. The aim of our study was to offset these limitations. Methods We developed a structured, 15-item objective structured clinical examination (OSCE)-type checklist of discrete teaching behaviours intended to be: (i) observable; (ii) applicable to multiple disciplines, and (iii) reliably identifiable. Our goal was to test and utilise this checklist as an objective assessment of clinical teaching across a range of in-patient teaching rounds experiences. During 2007-2008, pairs of external raters on two separate occasions observed nine attending physicians during actual in-patient paediatrics and internal medicine ward rounds at a large, academic medical centre. Observers documented the extent to which specific teaching behaviours did or did not occur. Results The internal consistency of the 15-item checklist was good (α = 0.85). A two-facet, partially nested G study found the generalisability of ratings to be generally acceptable, but inter-rater reliability varied greatly between occasions and across individual checklist items. Conclusions Despite attempts to identify discrete and observable target behaviours, placing observers on rounds to detect these behaviours may not be as straightforward as it would seem. Clinical teaching may be a more inherently subjective process, based on different teaching styles of faculty staff. However, a set of objective checklist items to be completed by trained observers on teaching rounds holds promise as a potentially viable means of identifying strengths and weaknesses of clinical instruction. Further research is needed to define what constitutes quality clinical teaching, as well as the most reliable method for assessing it.

Original languageEnglish (US)
Pages (from-to)379-386
Number of pages8
JournalMedical Education
Volume44
Issue number4
DOIs
StatePublished - Apr 2010
Externally publishedYes

Fingerprint

Teaching
Checklist
teacher
Teaching Rounds
Epidemiologic Effect Modifiers
educational content
teaching style
Internal Medicine
evaluation
Pediatrics
Physicians
rating
Education
physician
medicine
instruction
staff
examination
Research
experience

ASJC Scopus subject areas

  • Medicine(all)
  • Education

Cite this

Assessing the quality of clinical teaching : A preliminary study: Teacher assessment. / Conigliaro, Rosemarie L.; Stratton, Terry D.

In: Medical Education, Vol. 44, No. 4, 04.2010, p. 379-386.

Research output: Contribution to journalArticle

Conigliaro, Rosemarie L. ; Stratton, Terry D. / Assessing the quality of clinical teaching : A preliminary study: Teacher assessment. In: Medical Education. 2010 ; Vol. 44, No. 4. pp. 379-386.
@article{5d754bc4ce064b6b9c7bec01eb235583,
title = "Assessing the quality of clinical teaching: A preliminary study: Teacher assessment",
abstract = "Objectives Evaluations in the clinical arena are fraught with problems. Current assessments of clinical teaching typically measure attributes of clinical teachers in overly broad terms, are often subjective and often succumb to the halo effect. This is in contradistinction to measurements of lectures, workshops or online educational content, which can more readily be assessed using objective criteria. As a result, clinical evaluations are often insufficient to provide focused feedback, guide faculty development or identify specific areas for clinical teachers to implement change and improvement. The aim of our study was to offset these limitations. Methods We developed a structured, 15-item objective structured clinical examination (OSCE)-type checklist of discrete teaching behaviours intended to be: (i) observable; (ii) applicable to multiple disciplines, and (iii) reliably identifiable. Our goal was to test and utilise this checklist as an objective assessment of clinical teaching across a range of in-patient teaching rounds experiences. During 2007-2008, pairs of external raters on two separate occasions observed nine attending physicians during actual in-patient paediatrics and internal medicine ward rounds at a large, academic medical centre. Observers documented the extent to which specific teaching behaviours did or did not occur. Results The internal consistency of the 15-item checklist was good (α = 0.85). A two-facet, partially nested G study found the generalisability of ratings to be generally acceptable, but inter-rater reliability varied greatly between occasions and across individual checklist items. Conclusions Despite attempts to identify discrete and observable target behaviours, placing observers on rounds to detect these behaviours may not be as straightforward as it would seem. Clinical teaching may be a more inherently subjective process, based on different teaching styles of faculty staff. However, a set of objective checklist items to be completed by trained observers on teaching rounds holds promise as a potentially viable means of identifying strengths and weaknesses of clinical instruction. Further research is needed to define what constitutes quality clinical teaching, as well as the most reliable method for assessing it.",
author = "Conigliaro, {Rosemarie L.} and Stratton, {Terry D.}",
year = "2010",
month = "4",
doi = "10.1111/j.1365-2923.2009.03612.x",
language = "English (US)",
volume = "44",
pages = "379--386",
journal = "Medical Education",
issn = "0308-0110",
publisher = "Wiley-Blackwell",
number = "4",

}

TY - JOUR

T1 - Assessing the quality of clinical teaching

T2 - A preliminary study: Teacher assessment

AU - Conigliaro, Rosemarie L.

AU - Stratton, Terry D.

PY - 2010/4

Y1 - 2010/4

N2 - Objectives Evaluations in the clinical arena are fraught with problems. Current assessments of clinical teaching typically measure attributes of clinical teachers in overly broad terms, are often subjective and often succumb to the halo effect. This is in contradistinction to measurements of lectures, workshops or online educational content, which can more readily be assessed using objective criteria. As a result, clinical evaluations are often insufficient to provide focused feedback, guide faculty development or identify specific areas for clinical teachers to implement change and improvement. The aim of our study was to offset these limitations. Methods We developed a structured, 15-item objective structured clinical examination (OSCE)-type checklist of discrete teaching behaviours intended to be: (i) observable; (ii) applicable to multiple disciplines, and (iii) reliably identifiable. Our goal was to test and utilise this checklist as an objective assessment of clinical teaching across a range of in-patient teaching rounds experiences. During 2007-2008, pairs of external raters on two separate occasions observed nine attending physicians during actual in-patient paediatrics and internal medicine ward rounds at a large, academic medical centre. Observers documented the extent to which specific teaching behaviours did or did not occur. Results The internal consistency of the 15-item checklist was good (α = 0.85). A two-facet, partially nested G study found the generalisability of ratings to be generally acceptable, but inter-rater reliability varied greatly between occasions and across individual checklist items. Conclusions Despite attempts to identify discrete and observable target behaviours, placing observers on rounds to detect these behaviours may not be as straightforward as it would seem. Clinical teaching may be a more inherently subjective process, based on different teaching styles of faculty staff. However, a set of objective checklist items to be completed by trained observers on teaching rounds holds promise as a potentially viable means of identifying strengths and weaknesses of clinical instruction. Further research is needed to define what constitutes quality clinical teaching, as well as the most reliable method for assessing it.

AB - Objectives Evaluations in the clinical arena are fraught with problems. Current assessments of clinical teaching typically measure attributes of clinical teachers in overly broad terms, are often subjective and often succumb to the halo effect. This is in contradistinction to measurements of lectures, workshops or online educational content, which can more readily be assessed using objective criteria. As a result, clinical evaluations are often insufficient to provide focused feedback, guide faculty development or identify specific areas for clinical teachers to implement change and improvement. The aim of our study was to offset these limitations. Methods We developed a structured, 15-item objective structured clinical examination (OSCE)-type checklist of discrete teaching behaviours intended to be: (i) observable; (ii) applicable to multiple disciplines, and (iii) reliably identifiable. Our goal was to test and utilise this checklist as an objective assessment of clinical teaching across a range of in-patient teaching rounds experiences. During 2007-2008, pairs of external raters on two separate occasions observed nine attending physicians during actual in-patient paediatrics and internal medicine ward rounds at a large, academic medical centre. Observers documented the extent to which specific teaching behaviours did or did not occur. Results The internal consistency of the 15-item checklist was good (α = 0.85). A two-facet, partially nested G study found the generalisability of ratings to be generally acceptable, but inter-rater reliability varied greatly between occasions and across individual checklist items. Conclusions Despite attempts to identify discrete and observable target behaviours, placing observers on rounds to detect these behaviours may not be as straightforward as it would seem. Clinical teaching may be a more inherently subjective process, based on different teaching styles of faculty staff. However, a set of objective checklist items to be completed by trained observers on teaching rounds holds promise as a potentially viable means of identifying strengths and weaknesses of clinical instruction. Further research is needed to define what constitutes quality clinical teaching, as well as the most reliable method for assessing it.

UR - http://www.scopus.com/inward/record.url?scp=77949904032&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77949904032&partnerID=8YFLogxK

U2 - 10.1111/j.1365-2923.2009.03612.x

DO - 10.1111/j.1365-2923.2009.03612.x

M3 - Article

C2 - 20444073

AN - SCOPUS:77949904032

VL - 44

SP - 379

EP - 386

JO - Medical Education

JF - Medical Education

SN - 0308-0110

IS - 4

ER -