Between the headline ("The Glory of the Cheat Day") and the lead photo (above), I really wanted this article in The Atlantic to be beyond reproach. The article asserts that a "recent study" found that people that allowed themselves to occasionally cheat on their goal (like a diet) had a greater likelihood of success. The by-line is very official: "A new study suggests that planned lapses in self-control can help you stick with your goals over time." They even cite the flashy term that the study gives cheating and italicize it for emphasis: "planned hedonic deviations." That is so official, it has to be true!
As always, digging a little deeper shows that the conclusion might be too good to be true. The article does state that the study relied on three experiments:
- The first experiment was a thought exercises for the participants. As we all know after reading Everydata, self-reported data can be biased. The article doesn't tell us the sample size.
- The second experiment puts 36 people through the diets envisioned during the thought exercise in the first experiment. It's a small number of people and they were only on the diets for two weeks.
- The third experiment is a questionnaire asking people how they might react. Again, self-reported data.
Relying on the link in the article to slightly more detailed summary (though equally glutinous image) in the blog Research Digest by the British Psychological Society, we learn that the study used only 159 people across the three experiments: 59 in the first, 36 as stated in the second, and 64 in the third. All very small groups. We also learn that in the second study, the participants are given one cheat day per week so the affects are measured over two events.
While the results may be accurate, we might just have to struggle on with our diets until deeper research can be done.