Product & Services > One Care Street > Research Studies > 

One Care Street™ - Research Studies (ROI, efficacy, etc.)

Research Abstract summarizing the impact of using One Care Street tm in a Fortune 100 company

Prospective Targeting of Potential High-Utilizers for Health Coaching:

Results of a Pilot Study of Pre-Medicare Retirees


Objectives: (1) To determine the accuracy of a predictive model, using self-reported health survey data, to prospectively identify high near-term utilizers of health care. (2) To examine the effect of health coaching of these high utilizing individuals on their subsequent health care claims at 6 and 12 months.

Design: Pre/post prospective cohort study using propensity scores to adjust for biases inherent in high risk patients’ agreeing to participate in One Care StreetTM (OCS) and its health coaching intervention.

Setting: In spring 2002, a Fortune 100 employer with a large national managed care organization funded The Haelan Group (Haelan) to perform a pilot study assessing the potential for improved health and lowered cost through the use of OCS. OCS is a population health management system, that utilizes proprietary predictive modeling (based on demographic data and personal health perceptions and behaviors) and highly individualized, proactive health coaching for individuals at high risk for generating higher costs. The pilot study included 5,127 pre-Medicare retirees, then insured through the employer with traditional administrative and care coordination services provided by the health plan. The study population became eligible to take the OCS survey in 7 deployment groups from June through December of 2002 with subsequent enrollment of high-risk individuals into the health-coaching program as they were identified. A division of the health plan provided the health coaches (all registered nurses) for the pilot program. The coaches utilized the OCS coaching models and software system. The final cohorts of continuously-enrolled individuals for whom both OCS and claims data were available included 1,692 who had cost data available for the 12 months before and 6 months after OCS screening, and 1,556 who had full claims data for 12 months before and after screening.

Measures: All eligible subjects were asked to complete the OCS survey, a 45-item health perception assessment either online or on paper as each deployment group was notified between June and December of 2002. Claims data were subsequently obtained from the health plan and pharmacy-benefit manager for the entire 24 month study period. The final study cohort was determined by including all those with both OCS and continuous claims data during the 12 months prior to each person’s participation in OCS and the 12 months after (determined as each person’s “Index Date”). All claims dollars were indexed to the last month of each subject’s 24-month study period to control for inflation. Therefore, regardless of the person’s Index Date, data could be aggregated across all participants into the 12-month pre- and post-periods. Outliers, defined as those individuals with greater than $50,000 in either the pre- (N=12) or post-periods (N=17) were left in for descriptive analyses but were excluded from the analysis of the intervention’s effect.

Analysis: To determine the accuracy of the OCS predictive model in identifying high-care users, the predicted status of each individual (high or low-risk for being a high utilizer as determined by OCS risk score) was compared to their observed status (high = 20% highest costs; low = 80% lowest costs) at 6 months post-screening among the 1,692 individuals with 6-months post-screening claims. Additional analyses (which will be reported separately) compared the additional predictive power gained by combining OCS’ variables and the health plan’s claims-based predictive model. All tentative models then underwent a business analysis to determine the model which best optimizes highest accuracy at the lowest identification cost going forward.

To determine the intervention effect, we first established an equivalent control group of subjects determined by OCS to be high-risk but were not coached. To accomplish this, we developed a propensity score of those variables that discriminated between subjects coached and not coached. The subsequent intervention effect was then calculated by determining the difference in costs between the pre-12 months and the post-12 months total costs between the treatment and control groups, adjusting for treatment bias via the propensity score. Further, we determined the effect of the number of coaching sessions on treatment effect by comparing subjects who received 1-3 coaching sessions with those who received 4 or more sessions.

Results: Predictive accuracy: Though the original OCS predictive model (at a 0.5 risk score threshold with 77% sensitivity) added differential value to the health plan’s predictive model, the false positive rate was unacceptably high at 58%. A re-derived OCS model for this particular age group (i.e., re-weighting the variables for this subject population) yielded a sensitivity of 57% and specificity of 79%, thus lowering the false positive rate to more acceptable levels and still identifying more than half of the subjects who were in the top 20% group of post-screening costs.

Intervention effect: Analyzing the 1,556 who had the full 12 months of pre- and post-claims data, 612 (39%) were in the OCS low-risk group and 944 (61%) were in the OCS high-risk group. Among high-risk subjects, 556 (59%) were coached. The most common reasons for not coaching subjects were the inability to contact them, subject refusal, subject’s issues were resolved, and subject already felt he/she had adequate resources for dealing with his/her health care issues. Of the 556 who were coached, 64 (12%) had 4+ coaching sessions while 492 (88%) had 1-3 sessions. To better control for the regression-to-the-mean bias inherent in a pre/post design, an equivalent control group (adjusted for treatment bias using a propensity score) was established to determine the effect of the coaching intervention on pre- to post-12 month claims cost. The propensity score (derived using all OCS, demographic, and claims data as potential entering variables) had an acceptable c-statistic (ROC curve area) of 0.74. Using this control group, the coached high-risk group experienced an average $371 greater decrease in total costs comparing pre- to the post-12 month periods than the non-coached high-risk group. Those who received 1-3 sessions averaged a cost decease of $300 while those who received 4+ sessions had an average cost decrease of $2,251.

Conclusions: Self-reported factors improve the ability to predict an adult’s probability of becoming a near-term high care utilizer. Once identified, the cost of care for these high-care users can be positively impacted through an individualized coaching intervention. Going forward, effects can be enhanced by improving response rate, using the re-derived OCS model, engaging a higher percentage of high-risk into 4+ coaching sessions.

Respectfully submitted,

William M. Tierney, M.D.
Chancellor’s Professor of Medicine and Director,
Division of General Internal Medicine and Geriatrics
Indiana University School of Medicine
Senior Research Scientist, Regenstrief Institute, Incorporated