This post was updated May 19 at 11:22 p.m.

Researchers from UCLA and Carnegie Mellon University found that simple reminders were most effective in encouraging people to get their COVID-19 booster shots.

According to the study, published in March, the researchers aimed to assess insights from behavioral science that could transfer to real-world situations. They found that some text message reminders increased COVID-19 booster shots by between 1 and 2%, but other interventions favored by experts – such as bundling the COVID-19 booster with the flu vaccine – did not meaningfully increase booster uptake.

The researchers conducted three experimental trials involving more than 300,000 UCLA Health patients eligible for the COVID-19 booster shot – comparing the effects of interventions already tested in the field, interventions supported by hypothetical online studies and interventions supported by behavioral science.

One intervention the team evaluated was the concept of psychological ownership, which theorizes that people value something more when they feel ownership over it, said Hengchen Dai, one of the study’s co-first authors. To test this in the real world, the researchers sent text messages framing the vaccine as a dose reserved for the patient, rather than generally available, said Silvia Saccardo, a co-first author of the study and an associate professor of social and decision sciences at CMU.

Patients who received text messages were around 1% more likely to get the COVID-19 booster compared to those who did not. While the effect size was modest, increasing vaccination rates by even a small amount can have a meaningful impact, said Sitaram Vangala, one of the study’s co-authors and a principal statistician.

“If you think about half a million patients, that’s a lot,” said Vangala, the associate director of the UCLA Department of Medicine Statistics Core. “That’s a lot of patients who are getting vaccinated, which is a good thing.”

Factors such as specific patient populations and the timing of COVID-19 booster reminders can also impact how behavioral nudges perform in practice, Vangala said. For example, people were less likely to take action if reminders were sent during the evening or night, he said.

Vangala said even though psychological ownership language correlated to increased vaccination rates, it might not be effective at all times nor for all patients.

“Now we’re looking at boosters. We’re looking at a different time point. We’re looking at a period where people have received a lot more messaging about the COVID vaccine,” Vangala said. “But even this study doesn’t, I think, fully answer the question because we don’t necessarily know that this language is … going to generalize to other settings and other scenarios.”

These findings also underscore the need for more rigorous small-scale real-world testing of behavioral interventions before large-scale implementation, said Dai, an associate professor of management and organizations and behavioral decision-making at the UCLA Anderson School of Management. One reason for the discrepancy might be that people might pay more attention to online studies than they do in real life and may be influenced by wanting to give researchers positive results, she said.

In addition, since studies on vaccinations often rely on self-reporting, people may not follow through on their stated intentions, Saccardo said.

“Many times, researchers use intentions (and) self-reports, asking people, ‘Would you get vaccinated?’” Saccardo said. “Whatever has been found to be promising in those interventions – does it really affect behavior when deployed in the field?”

One limitation of the study is that it was only conducted within the UCLA Health system, Vangala said. For example, since not many UCLA patients are on Medicaid, the conclusions about interventions cannot be easily extrapolated to Medicaid-dominant populations, he added.

To meaningfully boost vaccination rates, future research needs to examine some of the root causes of vaccine hesitancy, such as how certain political ideologies discourage vaccination, Dai said. The researchers hope to explore ways to better tailor behavioral interventions to individuals’ specific attitudes towards vaccines, she added.

Saccardo added that it is important for policymakers to consider scientific research, including this study when looking at ways to promote vaccination.

“Don’t just base the interventions on intuition,” she said. “Research suggests that whatever works in hypotheticals might not actually transfer.”