Refining a Web-based goal assessment interview: item reduction based on reliability and predictive validity

Background: Goals are an important basis for patients’ cognitive appraisal processes underlying quality-of-life (QOL) assessment because they are the foundation to one’s frame of reference. We sought to identify the best of six goal delineation items and relevant themes for two new versions of the QOL Appraisal Profile: an interview tool using a subset of the best open-ended goal delineation items, and a shorter close-ended version for use in survey research. Methods: This is a secondary analysis of longitudinal data (n = 1126) of participants in the North American Research Committee on Multiple Sclerosis (MS) registry. The open-ended data were coded by at least two trained coders with moderately high inter-rater agreement. There were 31 themes reflecting goal content such as health, interpersonal, independence, mental health, and financial themes. Descriptive statistics identified most prevalent themes. Reliability analysis (alpha, item-total correlations) and hierarchical linear modeling identified the best goal items. Results: Based on these qualitative and quantitative analyses, Solve (item 2) is the best single item because it is clear anchor for about a third of the goal themes, and explains the most variance in outcomes and demographic characteristics, suggesting that it taps into and reveals diversity in the sample. The next best items are Accomplish and Maintain (items 1 and 4), which are useful in tapping into and revealing diversity among people reporting cognitive deficits (Accomplish), and demographic factors (both Accomplish and Maintain items). Conclusions: The goal delineation items identified as best performers in this study will be used to develop a shorter open-ended version of the QOL Appraisal Profile, and an entirely close-ended version of the QOL Appraisal Profile for use in more standard survey research settings. These tools will enable coaching patients in medical decision making as well as investigations of appraisal and response shift in QOL research.