Machine Learning Intervention Triples Serious Illness Conversations Among Oncology Clinicians

A stepped-wedge cluster randomized clinical trial published in JAMA Oncology indicated that an intervention that delivered machine learning mortality predictions with behavioral nudges to oncology clinicians significantly increased the rate of serious illness conversations (SIC) among all patients, especially among those with high mortality risk who were targeted by the intervention.¹

Based on these findings, researchers suggested behavioral nudges combined with machine learning mortality predictions can effectively influence clinician behavior and may be applied more broadly to enhance care for patients nearing the end of life.

“Within and outside of cancer, this is one of the first real-time applications of a machine learning algorithm paired with a prompt to actually help influence clinicians to initiate these discussions in a timely manner, before something unfortunate may happen,” co-lead author Ravi B. Parikh, MD, an assistant professor of Medical Ethics and Health Policy and Medicine in the Perelman School of Medicine at the University of Pennsylvania and a staff physician at the Corporal Michael J. Crescenz VA Medical Center, said in a press release.² “And it’s not just high-risk patients. It nearly doubled the number of conversations for patients who weren’t flagged—which tells us it’s eliciting a positive cultural change across the clinics to have more of these talks.”

The study was conducted across 20 weeks at 9 medical oncology clinics within a large academic health system in Pennsylvania, including 8 subspecialty oncology and 1 general oncology clinics. Clinicians at the 2 smallest subspecialty clinics were grouped together, which resulted in a total of 8 clinic groups randomly assigned to 4 intervention wedge periods.

The study sample consisted of a total of 78 clinicians and 14,607 patients. Interventions assessed in the study included weekly emails to oncology clinicians with SIC performance feedback and peer comparisons; a list of up to 6 high-risk patients (10% predicted risk of 180-day mortality) scheduled for the next week, estimated using a validated machine learning algorithm; and opt-out text message prompts to clinicians on a given patient’s appointment day to consider an SIC. Those included in the control group received usual care, consisting of weekly emails with cumulative SIC performance.

Across all patient encounters, SICs were conducted among 1.3% in the control group and 4.6% in the intervention group, which was deemed to be a significant difference (adjusted difference in percentage points, 3.3; 95% CI, 2.3-4.5; P < .001). Moreover, of 4124 high-risk patient encounters, SICs were conducted among 3.6% in the control group and 15.2% in the intervention group, which was also found to be a significant difference (adjusted difference in percentage points, 11.6; 95% CI, 8.2-12.5; P < .001).

“A key innovation of the present study was integrating behavioral economic principles into the [machine learning] based intervention, including peer comparison, performance feedback, and opt-out reminders, which have been used successfully in other clinical settings to change clinician behavior,” the authors noted. “Future implementation of artificial intelligence or [machine learning] tools in clinical practice should explore similar mechanisms of delivering predictions and integrating behavioral principles to maximally improve clinician and patient decision-making.”

Importantly though, this study was performed in a single academic health system, and the participants may not be representative of the general population of oncologists or patients with cancer. In addition, given that the study intervention consisted of several components, including a real-time prediction algorithm, a secure list of high-risk patients, peer comparisons, performance reports, and text messages, future work is necessary to assess the individual effects of each of these interventions.

“This is one of the first applications of combing behavioral nudges with machine learning methods in clinical care,” senior author Mitesh S. Patel, MD, director of the Penn Medicine Nudge Unit and an associate professor of Medicine in the Perelman School of Medicine at the University of Pennsylvania and a staff physician at the Corporal Michael J. Crescenz VA Medical Center, said in the release “There are many opportunities build upon this work and apply it to other aspects of cancer care and to other areas of medicine.”

Reference:

1. Manz C, Parikh RB, Small DS, et al. Effect of Integrating Machine Learning Mortality Estimates With Behavioral Nudges to Clinicians on Serious Illness Conversations Among Patients With Cancer. JAMA Oncology. doi: 10.1001/jamaoncol.2020.4759

2. Nudges Combined with Machine Learning Triples Advanced Care Conversations Among Patients with Cancer [news release]. Philadelphia. Published October 15, 2020. Accessed October 16, 2020. https://www.pennmedicine.org/news/news-releases/2020/october/nudges-combined-with-machine-learning-triples-advanced-care-conversations-among-patients-with-cancer