A continuation of our interview on innovation in testing what works to change healthcare outcomes.
With Dr. Scott Halpern, a practicing doctor, Director of the PAIR Center, and steering committee member of the Center for Health Incentives and Behavioral Economics at the University of Pennsylvania.
What’s driving change in health outcomes research?
Dr. Halpern: We need to improve knowledge about what works and what doesn’t. To change health-related behaviors, we need to improve the way we test and learn about the interventions that have the most impact at the best cost.
Too often, organizations have the very understandable impulse to just do something. Yet a more prudent response might be to take a step back and evaluate what works, but that has to happen at a faster pace than it did in the past.
How are you innovating with research to deliver insights faster?
Dr. Halpern: To drive this change, we are using what are called “Pragmatic Randomized Trials” which are trials with pragmatic designs. That’s in contrast to randomized control trials with ideologic or explanatory designs.
The distinction is that we’re looking to generate large-scale robust evidence of what works in the real world. Typically, most randomized trials are done to develop highly controlled evidence of what works under optimal or ideal circumstances. That often has very little resemblance to the real world.
The Pragmatic Randomized Trials approach leads us to test interventions that are already out there which may be used in haphazard ways. Also, there are interventions that we know can only help but certainly can’t hurt.
With that context, we have the opportunity to enroll people in the trials by default or using opt-out design. Where this approach is viable, we’ve had great success. By contrast, a typical randomized trial starts from the premise that people are not enrolled and they only get randomized if they proactively opt-in.
For example, a typical trial enrolls an average four or five patients a month and costs two or three thousand dollars per patient enrolled. Now, we’re conducting trials that enroll a thousand people a month at a cost of $40 or $50 per patient enrolled. It makes a big difference in the scale we can achieve.
How does this approach compare to common research practices?
Dr. Halpern: There’s more certainty with interventions that take place in more controlled circumstances. Yet insurers and employers want answers to real-world questions. With Pragmatic Randomized Trials, we’re often able to complete our trials in a much shorter time period than we otherwise would.
It’s more efficient than a sequential design in which we compare intervention A versus B, take that winner and compare it to C, and take the winner of that and compare it to D. If we could compare A, B, C, and D in the same trial because of the efficiencies in our recruitment strategies and using modern statistical methods — we can get larger amounts of data more quickly and at lower cost.
Can you give us an example from the e-cigarette study you recently published in the New England Journal of Medicine?
Dr. Halpern: I was just talking about A, B, C, and D. This trial compared five different approaches to promoting smoking cessation among people who work for 54 different companies around the United States.
We had a couple of innovations in this trial. One is the use of opt-out consent. People are enrolled automatically unless they opt out, which is exactly what would happen if an employer rolled out any of these interventions to its employees. In this particular study, we found that fewer than 2% of people chose not to participate.
We also had a big sample size and were able to test a ‘minimalist approach’ to smoking cessation. A minimalist approach is motivational text messaging and information. We compared that with providing free nicotine replacement therapy and free electronic cigarettes. Two additional arms of the trial used free nicotine replacement therapy plus two different ways of delivering financial incentives worth up to $600 dollars conditional on people stopping smoking. One used the cash as a pure carrot. You get the money if you succeed. Participants in the other incentive arm were told that they would lose money that was already theirs if they failed to stop smoking.
Among a group of 6,000 smokers, very few were motivated to quit. That represents the real population of employees. Very few stop smoking overall, but those in the trial that got the money quit at 3 times the rates of those in the other groups.
Importantly, we found that free provision of nicotine replacement therapy or free e-cigarettes didn’t really do anything at all. At least not compared to just merely providing information about the benefits of smoking cessation and text messaging.
What’s been the response to your findings?
Dr. Halpern: As you probably would guess, the reaction to this study has been mixed.
On the one hand, we’ve been lauded for conducting such a large trial that asked and answered a set of real world questions. On the other hand, we’ve been chastised by e-cigarette advocates for testing the effects of offering e-cigarettes not the effects of using e-cigarettes.
