2019-2020

Reinforcement Learning to Personalize Message Framing for Health Habits


Highly influential in motivating behavior change is how information is framed. However, the impact of framing is almost certainly highly individual and moreover difficult to identify or provide tailored framing at scale. Reinforcement learning (RL) is an advanced analytic method that discovers each individual’s pattern of responsiveness by observing their actions and then implements a personalized strategy to optimize behaviors. In contrast to other approaches using previously-collected data, RL algorithms learn in real-time and personalize for specific patients. This pilot study will have three phases: 1) patient interviews to help design the text messages, 2) a small pragmatic trial to determine the feasibility and effectiveness of using reinforcement learning on patient adherence to diabetes medications using the text messages designed in Phase 1, and 3) the identification of clusters of responsiveness to text messages using predictive analytics and post-hoc data from the trial. The trial’s proposed outcomes will be adherence (primary) and disease control. Pilot Study Team:

  • Julie Lauffenburger, PharmD, PhD, Assistant Professor at Harvard whose work focuses on optimizing medication use.
  • Punam Keller, PhD, Professor at Dartmouth, whose work focuses on health communication to patients and healthcare providers.
  • Elad Yom-Tov, PhD, principal researcher at Microsoft Research Labs with expertise in advanced analytics and prior experience applying RL to motivate patients with diabetes to exercise.
  • Marie McDonnell, MD, a practicing endocrinologist and Chief of the Diabetes Section in the Division of Endocrinology, Diabetes, and Hypertension at BWH and Harvard Medical School.
  • Niteesh Choudhry, MD, PhD, Professor at Harvard.




Use of Cues and Rewards in Patients with Arthritis and Rheumatic Disease to Improve Medication Adherence


Non-adherence to evidence-based prescription medications results in preventable morbidity and mortality for middle-aged and older adults. For example, in the case of arthritis, which is the most common cause of disability in the US and the 4th most common condition among Medicare beneficiaries, adherence to evidence-based treatments is extremely poor and contributes to racial/ethnic, socioeconomic, and gender disparities. Numerous interventions have been tested to help patients adhere to their prescribed therapies but even the most effective of these approaches have been only modestly effective and when removed, adherence often falls back to baseline. Taking medications intended for daily use, like those to prevent or treat chronic conditions, is a repetitive action that has great similarity with other behaviors that must be performed consistently, such as regular exercise, healthy eating and hand washing. In these cases, people who act consistently do so out of habit. Wendy Wood and colleagues have proposed that habit formation has three central components: behavioral repetition, associated context cues, and rewards. The principle-driven repetition-cue-reward model has obvious applicability to the daily repetitive activity of medication taking but has not been tested for this behavior nor adapted as an intervention for patients in real-world care settings. Accordingly, we will conduct a two-phase NIH Stage I pilot study that will adapt the “cue-reward” methodology to improve adherence to rheumatic disease medications. This pilot study will have two phases: 1) observation of adherence patterns and 2) intervention development and testing using a registry randomized cross-over trial. The trial’s proposed effectiveness outcome will be adherence, but we will also measure the acceptability, adoption, perceived appropriateness, fidelity, and feasibility of the intervention. Pilot Study Team:

  • Candace Feldman, MD, ScD, Assistant Professor at Harvard and Rheumatologist
  • Wendy Wood, PhD, Professor of Psychology and Business at the University of Southern California
  • Ted Robertson, Managing Director of ideas42
  • Niteesh Choudhry, MD, PhD, Professor at Harvard.





2020-2021

Personalizing Intervention to Reduce Clinical Inertia in the Treatment of Hypertension


More than half of patients with hypertension have blood pressures that are above clinical goal. The lack of intensification of hypertension treatment, often referred to as “clinical inertia”, is a large contributor to the suboptimal control of hypertension. Physicians appear to have different underlying reasons for clinical inertia and to also be influenced by individual patient characteristics and preferences. The goal of this study is to seek to empirically identify factors contributing to inertia and to identify the relationship between these factors and intervention responsiveness.

This pilot study will have two phases: 1) A 3-arm pragmatic randomized pilot study targeting primary care physicians caring for patients potentially in need of hypertension treatment intensification, 2) Identifying intervention responsiveness profiles through patient and physician clustering using a machine learning approach and interviews with select providers from each arm of the trial. The trial’s proposed outcomes will be intensification and changes in blood pressure.

Pilot Study Team:
  • Nancy Haff, MD, MPH, Instructor in Medicine at Harvard and primary care physician whose work focuses the implementation of evidence-based practices for cardiometabolic disease in primary care, clinical inertia, medication adherence, and health behavior change.
  • Wendy Wood, PhD, Professor of Psychology and Business at the University of Southern California
  • Elad Yom-Tov, PhD, principal researcher at Microsoft Research Labs with expertise in advanced analytics and prior experience applying RL to motivate patients with diabetes to exercise.
  • Niteesh Choudhry, MD, PhD, hospitalist at BWH and Professor of Medicine at Harvard




Application of Simulation for Providers to Overcome Decisional Gaps in High-risk Prescribing


Overuse of medications, particularly those with psychoactive properties such as antipsychotics, benzodiazepines, or sedative hypnotic “Z-drugs”, is common in inpatient and acute care settings to manage delirium and agitation despite the considerable associated risks. The goal of the proposed research is to develop and test a simulation-based intervention to address inappropriate prescribing of high-risk medications in inpatient settings by leveraging decisional gaps in System 1 and System 2 thinking, which may be driven through the hot-cold empathy gap (Figure). The “hot-cold” empathy gap is based on the observation that when people are in rational or “cold” states, they incorrectly predict what their behavior will be during “hot states.” This pilot study will have two phases: 1) interviews with providers and allied team members caring for older adults to help design the intervention and 2) a small pragmatic trial to determine the effectiveness of a simulation training-based intervention on the prescribing of high-risk medications. The trial’s proposed outcome will be the rate of high-risk medication prescribing. Study Team

  • Julie Lauffenburger, PharmD, PhD, Assistant Professor of Medicine at Harvard and inpatient pharmacist at BWH
  • Niteesh Choudhry, MD, PhD, hospitalist at BWH and Professor of Medicine at Harvard
  • Matthew DiFrancesco, MD, Assistant Director for Medicine at STRATUS, hospitalist at BWH and Instructor in Medicine at HMS
  • Maxwell Coll, MD, Internal Medicine Resident, BWH
  • Ted Robertson, MPA, Managing Director of ideas42, a behavioral economics design firm
  • Jerry Avorn, MD, geriatrician and Professor of Medicine at Harvard





Funded Pilot Projects