Refining a Web-based goal assessment interview: item reduction based on reliability and predictive validity

Carolyn E. Schwartz, Jei Li, Bruce D. Rapkin

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Background: Goals are an important basis for patients’ cognitive appraisal processes underlying quality-of-life (QOL) assessment because they are the foundation to one’s frame of reference. We sought to identify the best of six goal delineation items and relevant themes for two new versions of the QOL Appraisal Profile: an interview tool using a subset of the best open-ended goal delineation items, and a shorter close-ended version for use in survey research. Methods: This is a secondary analysis of longitudinal data (n = 1126) of participants in the North American Research Committee on Multiple Sclerosis (MS) registry. The open-ended data were coded by at least two trained coders with moderately high inter-rater agreement. There were 31 themes reflecting goal content such as health, interpersonal, independence, mental health, and financial themes. Descriptive statistics identified most prevalent themes. Reliability analysis (alpha, item-total correlations) and hierarchical linear modeling identified the best goal items. Results: Based on these qualitative and quantitative analyses, Solve (item 2) is the best single item because it is clear anchor for about a third of the goal themes, and explains the most variance in outcomes and demographic characteristics, suggesting that it taps into and reveals diversity in the sample. The next best items are Accomplish and Maintain (items 1 and 4), which are useful in tapping into and revealing diversity among people reporting cognitive deficits (Accomplish), and demographic factors (both Accomplish and Maintain items). Conclusions: The goal delineation items identified as best performers in this study will be used to develop a shorter open-ended version of the QOL Appraisal Profile, and an entirely close-ended version of the QOL Appraisal Profile for use in more standard survey research settings. These tools will enable coaching patients in medical decision making as well as investigations of appraisal and response shift in QOL research.

Original languageEnglish (US)
Pages (from-to)1-12
Number of pages12
JournalQuality of Life Research
DOIs
StateAccepted/In press - Mar 10 2016

Fingerprint

Reproducibility of Results
Interviews
Quality of Life
Research
Demography
Multiple Sclerosis
Registries
Mental Health
Health

Keywords

  • Appraisal
  • Goal assessment
  • Multiple sclerosis
  • Patient-reported outcomes
  • Response shift

ASJC Scopus subject areas

  • Public Health, Environmental and Occupational Health

Cite this

Refining a Web-based goal assessment interview : item reduction based on reliability and predictive validity. / Schwartz, Carolyn E.; Li, Jei; Rapkin, Bruce D.

In: Quality of Life Research, 10.03.2016, p. 1-12.

Research output: Contribution to journalArticle

@article{21222fbd88924191aa4509d5a4ede206,
title = "Refining a Web-based goal assessment interview: item reduction based on reliability and predictive validity",
abstract = "Background: Goals are an important basis for patients’ cognitive appraisal processes underlying quality-of-life (QOL) assessment because they are the foundation to one’s frame of reference. We sought to identify the best of six goal delineation items and relevant themes for two new versions of the QOL Appraisal Profile: an interview tool using a subset of the best open-ended goal delineation items, and a shorter close-ended version for use in survey research. Methods: This is a secondary analysis of longitudinal data (n = 1126) of participants in the North American Research Committee on Multiple Sclerosis (MS) registry. The open-ended data were coded by at least two trained coders with moderately high inter-rater agreement. There were 31 themes reflecting goal content such as health, interpersonal, independence, mental health, and financial themes. Descriptive statistics identified most prevalent themes. Reliability analysis (alpha, item-total correlations) and hierarchical linear modeling identified the best goal items. Results: Based on these qualitative and quantitative analyses, Solve (item 2) is the best single item because it is clear anchor for about a third of the goal themes, and explains the most variance in outcomes and demographic characteristics, suggesting that it taps into and reveals diversity in the sample. The next best items are Accomplish and Maintain (items 1 and 4), which are useful in tapping into and revealing diversity among people reporting cognitive deficits (Accomplish), and demographic factors (both Accomplish and Maintain items). Conclusions: The goal delineation items identified as best performers in this study will be used to develop a shorter open-ended version of the QOL Appraisal Profile, and an entirely close-ended version of the QOL Appraisal Profile for use in more standard survey research settings. These tools will enable coaching patients in medical decision making as well as investigations of appraisal and response shift in QOL research.",
keywords = "Appraisal, Goal assessment, Multiple sclerosis, Patient-reported outcomes, Response shift",
author = "Schwartz, {Carolyn E.} and Jei Li and Rapkin, {Bruce D.}",
year = "2016",
month = "3",
day = "10",
doi = "10.1007/s11136-016-1258-x",
language = "English (US)",
pages = "1--12",
journal = "Quality of Life Research",
issn = "0962-9343",
publisher = "Springer Netherlands",

}

TY - JOUR

T1 - Refining a Web-based goal assessment interview

T2 - item reduction based on reliability and predictive validity

AU - Schwartz, Carolyn E.

AU - Li, Jei

AU - Rapkin, Bruce D.

PY - 2016/3/10

Y1 - 2016/3/10

N2 - Background: Goals are an important basis for patients’ cognitive appraisal processes underlying quality-of-life (QOL) assessment because they are the foundation to one’s frame of reference. We sought to identify the best of six goal delineation items and relevant themes for two new versions of the QOL Appraisal Profile: an interview tool using a subset of the best open-ended goal delineation items, and a shorter close-ended version for use in survey research. Methods: This is a secondary analysis of longitudinal data (n = 1126) of participants in the North American Research Committee on Multiple Sclerosis (MS) registry. The open-ended data were coded by at least two trained coders with moderately high inter-rater agreement. There were 31 themes reflecting goal content such as health, interpersonal, independence, mental health, and financial themes. Descriptive statistics identified most prevalent themes. Reliability analysis (alpha, item-total correlations) and hierarchical linear modeling identified the best goal items. Results: Based on these qualitative and quantitative analyses, Solve (item 2) is the best single item because it is clear anchor for about a third of the goal themes, and explains the most variance in outcomes and demographic characteristics, suggesting that it taps into and reveals diversity in the sample. The next best items are Accomplish and Maintain (items 1 and 4), which are useful in tapping into and revealing diversity among people reporting cognitive deficits (Accomplish), and demographic factors (both Accomplish and Maintain items). Conclusions: The goal delineation items identified as best performers in this study will be used to develop a shorter open-ended version of the QOL Appraisal Profile, and an entirely close-ended version of the QOL Appraisal Profile for use in more standard survey research settings. These tools will enable coaching patients in medical decision making as well as investigations of appraisal and response shift in QOL research.

AB - Background: Goals are an important basis for patients’ cognitive appraisal processes underlying quality-of-life (QOL) assessment because they are the foundation to one’s frame of reference. We sought to identify the best of six goal delineation items and relevant themes for two new versions of the QOL Appraisal Profile: an interview tool using a subset of the best open-ended goal delineation items, and a shorter close-ended version for use in survey research. Methods: This is a secondary analysis of longitudinal data (n = 1126) of participants in the North American Research Committee on Multiple Sclerosis (MS) registry. The open-ended data were coded by at least two trained coders with moderately high inter-rater agreement. There were 31 themes reflecting goal content such as health, interpersonal, independence, mental health, and financial themes. Descriptive statistics identified most prevalent themes. Reliability analysis (alpha, item-total correlations) and hierarchical linear modeling identified the best goal items. Results: Based on these qualitative and quantitative analyses, Solve (item 2) is the best single item because it is clear anchor for about a third of the goal themes, and explains the most variance in outcomes and demographic characteristics, suggesting that it taps into and reveals diversity in the sample. The next best items are Accomplish and Maintain (items 1 and 4), which are useful in tapping into and revealing diversity among people reporting cognitive deficits (Accomplish), and demographic factors (both Accomplish and Maintain items). Conclusions: The goal delineation items identified as best performers in this study will be used to develop a shorter open-ended version of the QOL Appraisal Profile, and an entirely close-ended version of the QOL Appraisal Profile for use in more standard survey research settings. These tools will enable coaching patients in medical decision making as well as investigations of appraisal and response shift in QOL research.

KW - Appraisal

KW - Goal assessment

KW - Multiple sclerosis

KW - Patient-reported outcomes

KW - Response shift

UR - http://www.scopus.com/inward/record.url?scp=84960406064&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84960406064&partnerID=8YFLogxK

U2 - 10.1007/s11136-016-1258-x

DO - 10.1007/s11136-016-1258-x

M3 - Article

C2 - 26961007

AN - SCOPUS:84960406064

SP - 1

EP - 12

JO - Quality of Life Research

JF - Quality of Life Research

SN - 0962-9343

ER -