Reliability and Validity of Ambulatory Cognitive Assessments

Martin J. Sliwinski, Jacqueline A. Mogle, Jinshil Hyun, Elizabeth Munoz, Joshua M. Smyth, Richard B. Lipton

Research output: Contribution to journalArticle

22 Citations (Scopus)

Abstract

Mobile technologies are increasingly used to measure cognitive function outside of traditional clinic and laboratory settings. Although ambulatory assessments of cognitive function conducted in people’s natural environments offer potential advantages over traditional assessment approaches, the psychometrics of cognitive assessment procedures have been understudied. We evaluated the reliability and construct validity of ambulatory assessments of working memory and perceptual speed administered via smartphones as part of an ecological momentary assessment protocol in a diverse adult sample (N = 219). Results indicated excellent between-person reliability (≥0.97) for average scores, and evidence of reliable within-person variability across measurement occasions (0.41-0.53). The ambulatory tasks also exhibited construct validity, as evidence by their loadings on working memory and perceptual speed factors defined by the in-lab assessments. Our findings demonstrate that averaging across brief cognitive assessments made in uncontrolled naturalistic settings provide measurements that are comparable in reliability to assessments made in controlled laboratory environments.

Original languageEnglish (US)
Pages (from-to)14-30
Number of pages17
JournalAssessment
Volume25
Issue number1
DOIs
StatePublished - Jan 1 2018

Fingerprint

Short-Term Memory
Reproducibility of Results
Cognition
Controlled Environment
Psychometrics
Technology
Smartphone
Ecological Momentary Assessment

Keywords

  • ambulatory assessment
  • cognition
  • ecological validity
  • smartphone

ASJC Scopus subject areas

  • Clinical Psychology
  • Applied Psychology

Cite this

Sliwinski, M. J., Mogle, J. A., Hyun, J., Munoz, E., Smyth, J. M., & Lipton, R. B. (2018). Reliability and Validity of Ambulatory Cognitive Assessments. Assessment, 25(1), 14-30. https://doi.org/10.1177/1073191116643164

Reliability and Validity of Ambulatory Cognitive Assessments. / Sliwinski, Martin J.; Mogle, Jacqueline A.; Hyun, Jinshil; Munoz, Elizabeth; Smyth, Joshua M.; Lipton, Richard B.

In: Assessment, Vol. 25, No. 1, 01.01.2018, p. 14-30.

Research output: Contribution to journalArticle

Sliwinski, MJ, Mogle, JA, Hyun, J, Munoz, E, Smyth, JM & Lipton, RB 2018, 'Reliability and Validity of Ambulatory Cognitive Assessments', Assessment, vol. 25, no. 1, pp. 14-30. https://doi.org/10.1177/1073191116643164
Sliwinski, Martin J. ; Mogle, Jacqueline A. ; Hyun, Jinshil ; Munoz, Elizabeth ; Smyth, Joshua M. ; Lipton, Richard B. / Reliability and Validity of Ambulatory Cognitive Assessments. In: Assessment. 2018 ; Vol. 25, No. 1. pp. 14-30.
@article{54b430d54a8e4f6db6ebe8c2c972d260,
title = "Reliability and Validity of Ambulatory Cognitive Assessments",
abstract = "Mobile technologies are increasingly used to measure cognitive function outside of traditional clinic and laboratory settings. Although ambulatory assessments of cognitive function conducted in people’s natural environments offer potential advantages over traditional assessment approaches, the psychometrics of cognitive assessment procedures have been understudied. We evaluated the reliability and construct validity of ambulatory assessments of working memory and perceptual speed administered via smartphones as part of an ecological momentary assessment protocol in a diverse adult sample (N = 219). Results indicated excellent between-person reliability (≥0.97) for average scores, and evidence of reliable within-person variability across measurement occasions (0.41-0.53). The ambulatory tasks also exhibited construct validity, as evidence by their loadings on working memory and perceptual speed factors defined by the in-lab assessments. Our findings demonstrate that averaging across brief cognitive assessments made in uncontrolled naturalistic settings provide measurements that are comparable in reliability to assessments made in controlled laboratory environments.",
keywords = "ambulatory assessment, cognition, ecological validity, smartphone",
author = "Sliwinski, {Martin J.} and Mogle, {Jacqueline A.} and Jinshil Hyun and Elizabeth Munoz and Smyth, {Joshua M.} and Lipton, {Richard B.}",
year = "2018",
month = "1",
day = "1",
doi = "10.1177/1073191116643164",
language = "English (US)",
volume = "25",
pages = "14--30",
journal = "Assessment",
issn = "1073-1911",
publisher = "SAGE Publications Inc.",
number = "1",

}

TY - JOUR

T1 - Reliability and Validity of Ambulatory Cognitive Assessments

AU - Sliwinski, Martin J.

AU - Mogle, Jacqueline A.

AU - Hyun, Jinshil

AU - Munoz, Elizabeth

AU - Smyth, Joshua M.

AU - Lipton, Richard B.

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Mobile technologies are increasingly used to measure cognitive function outside of traditional clinic and laboratory settings. Although ambulatory assessments of cognitive function conducted in people’s natural environments offer potential advantages over traditional assessment approaches, the psychometrics of cognitive assessment procedures have been understudied. We evaluated the reliability and construct validity of ambulatory assessments of working memory and perceptual speed administered via smartphones as part of an ecological momentary assessment protocol in a diverse adult sample (N = 219). Results indicated excellent between-person reliability (≥0.97) for average scores, and evidence of reliable within-person variability across measurement occasions (0.41-0.53). The ambulatory tasks also exhibited construct validity, as evidence by their loadings on working memory and perceptual speed factors defined by the in-lab assessments. Our findings demonstrate that averaging across brief cognitive assessments made in uncontrolled naturalistic settings provide measurements that are comparable in reliability to assessments made in controlled laboratory environments.

AB - Mobile technologies are increasingly used to measure cognitive function outside of traditional clinic and laboratory settings. Although ambulatory assessments of cognitive function conducted in people’s natural environments offer potential advantages over traditional assessment approaches, the psychometrics of cognitive assessment procedures have been understudied. We evaluated the reliability and construct validity of ambulatory assessments of working memory and perceptual speed administered via smartphones as part of an ecological momentary assessment protocol in a diverse adult sample (N = 219). Results indicated excellent between-person reliability (≥0.97) for average scores, and evidence of reliable within-person variability across measurement occasions (0.41-0.53). The ambulatory tasks also exhibited construct validity, as evidence by their loadings on working memory and perceptual speed factors defined by the in-lab assessments. Our findings demonstrate that averaging across brief cognitive assessments made in uncontrolled naturalistic settings provide measurements that are comparable in reliability to assessments made in controlled laboratory environments.

KW - ambulatory assessment

KW - cognition

KW - ecological validity

KW - smartphone

UR - http://www.scopus.com/inward/record.url?scp=85035046781&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85035046781&partnerID=8YFLogxK

U2 - 10.1177/1073191116643164

DO - 10.1177/1073191116643164

M3 - Article

C2 - 27084835

AN - SCOPUS:85035046781

VL - 25

SP - 14

EP - 30

JO - Assessment

JF - Assessment

SN - 1073-1911

IS - 1

ER -