Super learner analysis of electronic adherence data improves viral prediction and may provide strategies for selective HIV RNA monitoring

Maya L. Petersen, Erin LeDell, Joshua Schwab, Varada Sarovar, Robert Gross, Nancy Reynolds, Jessica E. Haberer, Kathy Goggin, Carol Golin, Julia H. Arnsten, Marc I. Rosen, Robert H. Remien, David Etoori, Ira B. Wilson, Jane M. Simoni, Judith A. Erlen, Mark J. Van Der Laan, Honghu Liu, David R. Bangsberg

Research output: Contribution to journalArticle

17 Citations (Scopus)

Abstract

Objective: Regular HIV RNA testing for all HIV-positive patients on antiretroviral therapy (ART) is expensive and has low yield since most tests are undetectable. Selective testing of those at higher risk of failure may improve efficiency. We investigated whether a novel analysis of adherence data could correctly classify virological failure and potentially inform a selective testing strategy. Design: Multisite prospective cohort consortium. Methods: We evaluated longitudinal data on 1478 adult patients treated with ART and monitored using the Medication Event Monitoring System (MEMS) in 16 US cohorts contributing to the MACH14 consortium. Because the relationship between adherence and virological failure is complex and heterogeneous, we applied a machine-learning algorithm (Super Learner) to build a model for classifying failure and evaluated its performance using cross-validation. Results: Application of the Super Learner algorithm to MEMS data, combined with data on CD4<sup>+</sup> T-cell counts and ART regimen, significantly improved classification of virological failure over a single MEMS adherence measure. Area under the receiver operating characteristic curve, evaluated on data not used in model fitting, was 0.78 (95% confidence interval: 0.75 to 0.80) and 0.79 (95% confidence interval: 0.76 to 0.81) for failure defined as single HIV RNA level >1000 copies per milliliter or >400 copies per milliliter, respectively. Our results suggest that 25%-31% of viral load tests could be avoided while maintaining sensitivity for failure detection at or above 95%, for a cost savings of $16-$29 per person-month. Conclusions: Our findings provide initial proof of concept for the potential use of electronic medication adherence data to reduce costs through behavior-driven HIV RNA testing.

Original languageEnglish (US)
Pages (from-to)109-118
Number of pages10
JournalJournal of Acquired Immune Deficiency Syndromes
Volume69
Issue number1
DOIs
StatePublished - May 1 2015

Fingerprint

HIV
RNA
Confidence Intervals
Medication Adherence
Cost Savings
CD4 Lymphocyte Count
Viral Load
Information Systems
ROC Curve
Therapeutics
T-Lymphocytes
Costs and Cost Analysis
Machine Learning

Keywords

  • adherence
  • antiretroviral therapy
  • HIV
  • HIV RNA monitoring
  • Medication Event Monitoring System
  • Super Learner
  • virological failure

ASJC Scopus subject areas

  • Infectious Diseases
  • Pharmacology (medical)

Cite this

Super learner analysis of electronic adherence data improves viral prediction and may provide strategies for selective HIV RNA monitoring. / Petersen, Maya L.; LeDell, Erin; Schwab, Joshua; Sarovar, Varada; Gross, Robert; Reynolds, Nancy; Haberer, Jessica E.; Goggin, Kathy; Golin, Carol; Arnsten, Julia H.; Rosen, Marc I.; Remien, Robert H.; Etoori, David; Wilson, Ira B.; Simoni, Jane M.; Erlen, Judith A.; Van Der Laan, Mark J.; Liu, Honghu; Bangsberg, David R.

In: Journal of Acquired Immune Deficiency Syndromes, Vol. 69, No. 1, 01.05.2015, p. 109-118.

Research output: Contribution to journalArticle

Petersen, ML, LeDell, E, Schwab, J, Sarovar, V, Gross, R, Reynolds, N, Haberer, JE, Goggin, K, Golin, C, Arnsten, JH, Rosen, MI, Remien, RH, Etoori, D, Wilson, IB, Simoni, JM, Erlen, JA, Van Der Laan, MJ, Liu, H & Bangsberg, DR 2015, 'Super learner analysis of electronic adherence data improves viral prediction and may provide strategies for selective HIV RNA monitoring', Journal of Acquired Immune Deficiency Syndromes, vol. 69, no. 1, pp. 109-118. https://doi.org/10.1097/QAI.0000000000000548
Petersen, Maya L. ; LeDell, Erin ; Schwab, Joshua ; Sarovar, Varada ; Gross, Robert ; Reynolds, Nancy ; Haberer, Jessica E. ; Goggin, Kathy ; Golin, Carol ; Arnsten, Julia H. ; Rosen, Marc I. ; Remien, Robert H. ; Etoori, David ; Wilson, Ira B. ; Simoni, Jane M. ; Erlen, Judith A. ; Van Der Laan, Mark J. ; Liu, Honghu ; Bangsberg, David R. / Super learner analysis of electronic adherence data improves viral prediction and may provide strategies for selective HIV RNA monitoring. In: Journal of Acquired Immune Deficiency Syndromes. 2015 ; Vol. 69, No. 1. pp. 109-118.
@article{c97564ed99d842368b1b4eb558a3928f,
title = "Super learner analysis of electronic adherence data improves viral prediction and may provide strategies for selective HIV RNA monitoring",
abstract = "Objective: Regular HIV RNA testing for all HIV-positive patients on antiretroviral therapy (ART) is expensive and has low yield since most tests are undetectable. Selective testing of those at higher risk of failure may improve efficiency. We investigated whether a novel analysis of adherence data could correctly classify virological failure and potentially inform a selective testing strategy. Design: Multisite prospective cohort consortium. Methods: We evaluated longitudinal data on 1478 adult patients treated with ART and monitored using the Medication Event Monitoring System (MEMS) in 16 US cohorts contributing to the MACH14 consortium. Because the relationship between adherence and virological failure is complex and heterogeneous, we applied a machine-learning algorithm (Super Learner) to build a model for classifying failure and evaluated its performance using cross-validation. Results: Application of the Super Learner algorithm to MEMS data, combined with data on CD4+ T-cell counts and ART regimen, significantly improved classification of virological failure over a single MEMS adherence measure. Area under the receiver operating characteristic curve, evaluated on data not used in model fitting, was 0.78 (95{\%} confidence interval: 0.75 to 0.80) and 0.79 (95{\%} confidence interval: 0.76 to 0.81) for failure defined as single HIV RNA level >1000 copies per milliliter or >400 copies per milliliter, respectively. Our results suggest that 25{\%}-31{\%} of viral load tests could be avoided while maintaining sensitivity for failure detection at or above 95{\%}, for a cost savings of $16-$29 per person-month. Conclusions: Our findings provide initial proof of concept for the potential use of electronic medication adherence data to reduce costs through behavior-driven HIV RNA testing.",
keywords = "adherence, antiretroviral therapy, HIV, HIV RNA monitoring, Medication Event Monitoring System, Super Learner, virological failure",
author = "Petersen, {Maya L.} and Erin LeDell and Joshua Schwab and Varada Sarovar and Robert Gross and Nancy Reynolds and Haberer, {Jessica E.} and Kathy Goggin and Carol Golin and Arnsten, {Julia H.} and Rosen, {Marc I.} and Remien, {Robert H.} and David Etoori and Wilson, {Ira B.} and Simoni, {Jane M.} and Erlen, {Judith A.} and {Van Der Laan}, {Mark J.} and Honghu Liu and Bangsberg, {David R.}",
year = "2015",
month = "5",
day = "1",
doi = "10.1097/QAI.0000000000000548",
language = "English (US)",
volume = "69",
pages = "109--118",
journal = "Journal of Acquired Immune Deficiency Syndromes",
issn = "1525-4135",
publisher = "Lippincott Williams and Wilkins",
number = "1",

}

TY - JOUR

T1 - Super learner analysis of electronic adherence data improves viral prediction and may provide strategies for selective HIV RNA monitoring

AU - Petersen, Maya L.

AU - LeDell, Erin

AU - Schwab, Joshua

AU - Sarovar, Varada

AU - Gross, Robert

AU - Reynolds, Nancy

AU - Haberer, Jessica E.

AU - Goggin, Kathy

AU - Golin, Carol

AU - Arnsten, Julia H.

AU - Rosen, Marc I.

AU - Remien, Robert H.

AU - Etoori, David

AU - Wilson, Ira B.

AU - Simoni, Jane M.

AU - Erlen, Judith A.

AU - Van Der Laan, Mark J.

AU - Liu, Honghu

AU - Bangsberg, David R.

PY - 2015/5/1

Y1 - 2015/5/1

N2 - Objective: Regular HIV RNA testing for all HIV-positive patients on antiretroviral therapy (ART) is expensive and has low yield since most tests are undetectable. Selective testing of those at higher risk of failure may improve efficiency. We investigated whether a novel analysis of adherence data could correctly classify virological failure and potentially inform a selective testing strategy. Design: Multisite prospective cohort consortium. Methods: We evaluated longitudinal data on 1478 adult patients treated with ART and monitored using the Medication Event Monitoring System (MEMS) in 16 US cohorts contributing to the MACH14 consortium. Because the relationship between adherence and virological failure is complex and heterogeneous, we applied a machine-learning algorithm (Super Learner) to build a model for classifying failure and evaluated its performance using cross-validation. Results: Application of the Super Learner algorithm to MEMS data, combined with data on CD4+ T-cell counts and ART regimen, significantly improved classification of virological failure over a single MEMS adherence measure. Area under the receiver operating characteristic curve, evaluated on data not used in model fitting, was 0.78 (95% confidence interval: 0.75 to 0.80) and 0.79 (95% confidence interval: 0.76 to 0.81) for failure defined as single HIV RNA level >1000 copies per milliliter or >400 copies per milliliter, respectively. Our results suggest that 25%-31% of viral load tests could be avoided while maintaining sensitivity for failure detection at or above 95%, for a cost savings of $16-$29 per person-month. Conclusions: Our findings provide initial proof of concept for the potential use of electronic medication adherence data to reduce costs through behavior-driven HIV RNA testing.

AB - Objective: Regular HIV RNA testing for all HIV-positive patients on antiretroviral therapy (ART) is expensive and has low yield since most tests are undetectable. Selective testing of those at higher risk of failure may improve efficiency. We investigated whether a novel analysis of adherence data could correctly classify virological failure and potentially inform a selective testing strategy. Design: Multisite prospective cohort consortium. Methods: We evaluated longitudinal data on 1478 adult patients treated with ART and monitored using the Medication Event Monitoring System (MEMS) in 16 US cohorts contributing to the MACH14 consortium. Because the relationship between adherence and virological failure is complex and heterogeneous, we applied a machine-learning algorithm (Super Learner) to build a model for classifying failure and evaluated its performance using cross-validation. Results: Application of the Super Learner algorithm to MEMS data, combined with data on CD4+ T-cell counts and ART regimen, significantly improved classification of virological failure over a single MEMS adherence measure. Area under the receiver operating characteristic curve, evaluated on data not used in model fitting, was 0.78 (95% confidence interval: 0.75 to 0.80) and 0.79 (95% confidence interval: 0.76 to 0.81) for failure defined as single HIV RNA level >1000 copies per milliliter or >400 copies per milliliter, respectively. Our results suggest that 25%-31% of viral load tests could be avoided while maintaining sensitivity for failure detection at or above 95%, for a cost savings of $16-$29 per person-month. Conclusions: Our findings provide initial proof of concept for the potential use of electronic medication adherence data to reduce costs through behavior-driven HIV RNA testing.

KW - adherence

KW - antiretroviral therapy

KW - HIV

KW - HIV RNA monitoring

KW - Medication Event Monitoring System

KW - Super Learner

KW - virological failure

UR - http://www.scopus.com/inward/record.url?scp=84928390562&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84928390562&partnerID=8YFLogxK

U2 - 10.1097/QAI.0000000000000548

DO - 10.1097/QAI.0000000000000548

M3 - Article

C2 - 25942462

AN - SCOPUS:84928390562

VL - 69

SP - 109

EP - 118

JO - Journal of Acquired Immune Deficiency Syndromes

JF - Journal of Acquired Immune Deficiency Syndromes

SN - 1525-4135

IS - 1

ER -