Please note that the paper discussed in this post is due to be corrected and republished at a later date.
The amount of data about the effectiveness of behavioral science interventions is growing. A 2019 meta-analysis based on 100 published studies shows that 62% of nudge treatments yield statistically significant results. In the sample, defaults are most effective, and precommitment nudges are least effective. More recent (2020) research finds that unpublished RCTs conducted by nudge units in the U.S. also produce statistically significant impacts. In terms of nudges aimed specifically at eating, another study concludes that behaviorally-oriented nudges (e.g. changing plate sizes) work better than affectively-oriented nudges (e.g. attractive displays), which are in turn more impactful than cognitively-oriented ones (e.g. adding nutrition details).
A new meta-analysis published in PNAS by Stephanie Mertens, Mario Herberz, Ulf J. J. Hahnel, and Tobias Brosch provides more evidence about the effectiveness of nudges. Here’s a summary:
- The study is a meta-analysis of 455 effect sizes from 214 publications.
- The data reveal a statistically significant relationship between choice architecture interventions (nudges) and behavior.
- The observed small to medium effect size is comparable to that achieved with traditional interventions.
- The dispersion of effect sizes suggests that about 15% of interventions may backfire, i.e., reduce or reverse the desired behavior.
- ‘Decision structure’ nudges (e.g. defaults) worked better than those categorized as ‘decision information’ (e.g. descriptive social norms) and ‘decision assistance’ (e.g. commitment devices).
- The greater effectiveness of ‘decision structure’ interventions could potentially be due to 1) lower demand of those nudges on information processing and 2) lower susceptibility to individual differences in values and goals.
- Food choice nudges work particularly well, with effect sizes up to 2.5 times greater than those in other areas (health, environment, finance, prosocial, other). The smallest effects can be seen in the financial domain.
- The food vs finance differences may be due to relatively low vs high impact (behavioral costs and long-term consequences) of decisions, as well as different levels of habitualizations, in those domains.
- Most contextual study characteristics (location, target population, and experimental setting) did not make a significant difference to nudge effectiveness.
- Finally, the meta-analysis suggests a moderate publication bias (toward positive results, as we would expect). This means that nudges are likely to have smaller true effects than those estimated by the model.
The article by Mertens and her colleagues is another important addition to a growing body of research on nudge effectiveness. As more work is done in this area in the future, we also hope to gain insights from new questions. For example, how do different sociodemographic and psychological factors influence nudge effects? How does the effectiveness of nudges compare not only to that of traditional interventions, but also combinations of approaches?