In its October issue, Health magazine takes on the issue of dieting by doing “a study rigorous enough for a medical journal.” (It’s not currently available online.)

Well, they are right, it is “far beyond what magazines usually offer.” Alas, too bad it doesn’t really provide a satisfactory answer to the question of dieting.

First the premise. Health recruited 1,000 dieters and randomly assigned them to one of four different diets: Weight Watchers (the online version), Atkins, Dean Ornish, and David Katz’s Way to Eat. (Katz actually conducted the study, but the data was coded so he didn’t know until the end how his diet performed.)

In the article, Health suggests that Weight Watchers is the “winner.” However, what I think is the interesting stat (and the one that raises a whole set of questions re the relevance/usefulness of the study) is that only 16% of the participants were still in the study after 6 months. In fact, over 700 had dropped out after three months!

Umm, hello!?! More than 8 out of 10 dieters dropped out, and the story is that Weight Watchers is the winner?!?

To be fair, the magazine does not try to hide this. But this suggests to me that the rest of their “findings” are just a bit questionable. For example, the article dings Atkins’ reputation for helping with cravings, pointing out that Atkins “left 59% of our volunteers fighting frequent food cravings.” Well, given that only 17% of the Atkins dieters kept to the program most of the time, it’s hard not to wonder whether compliant dieters would have had less cravings.

A large part of the problem with the study was the fact that people were randomly assigned to diets. That’s really unfortunate. Atkins and Ornish are both fairly extreme (the former is very low-carb to start, the latter is very low fat and essentially a vegetarian diet). So it’s no surprise that 90% of the Atkins dieters and 85% of the Ornish dieters had dropped out (compared with 64% of the WW dieters and 67% of the Katz dieters).

Health claims that assigning diets to participants was meant to:

ensure the groups were similar at the start. If we’d allowed people to pick their own diets, we wouldn’t know how well the plans worked for the average person—only how they performed for those most inclined to try them.

Yet people would be far more likely to make a choice of a diet that had some level of appeal. In fact, they pointed out how the study required participants to just “go get the book” since that’s “real life.”

So the way it looks to me, in their quest to mimic real life and have good data, they set up a completely unrealistic situation (you don’t choose your eating plan) and wound up with questionable results.

That said, here are their findings:

1. Dieting is hard—really hard.
2. Some diets are easier to sustain.
3. The more restrictive the diet, the grumpier the dieter
4. Every diet can work.
5. It has to be easy enough to become automatic.

Weight Watchers and Katz’s diet scored higher on things like food choices, fitting in with families, and confidence for long-term compliance. Yet the question of which diet works “better” in terms of weight loss is obscured since so few dieters who were assigned those diets stuck with them. Interestingly, the dieter who lost the most weight in the 6 months followed Atkins (she lost 70lbs in the 6 months).

Now, I’m not necessarily defending Atkins. But I’m disappointed that this study didn’t do enough to explore the issue of carbs and weight loss, which is I think is a big issue re weight loss and compliance. I prefer the South Beach, Zone, and Protein Power versions of moderate carbs to Atkins, and think it might have been interesting had South Beach been chosen either in addition or instead of one of the others.

Since South Beach doesn’t cause quite the knee-jerk that Atkins does (in fact, the CSPI calls it a “healthy version of Atkins diet that’s backed by solid evidence on fats and heart disease”, pdf, p. 5 and there’s some reason for optimism in early studies), perhaps we’ll hear more in the future.

But ultimately, Health’s diet face-off seems to raise more questions than it answers. To be sure, motivated people may do quite well on Atkins or Ornish…or any diet for that matter. That’s what Oprah said late last season…it’s not the diets that are the problem. It’s the compliance that’s the issue.

And what Health discovered was that if you force people to do a diet they don’t want to do, their compliance will be terrible. No compliance, no success. No surprise there.

I do give them a lot of credit. After all, they spent a chunk of change on this (bought books or discounted WW online memberships for participants). And the problem with studies is that it is very hard to correct the course in the middle of the study (for example, once it was clear that people were bailing left and right).

But there’s just something troubling about the way they reported the results. After all, would you get on a plane if you only had a 16% change of actually getting to your destination? So I guess I just find it a bit bizarre that with so many people failing, they seem to just say “boy, dieting sucks big time, but we just don’t know what else to tell you.”

Now that’s a story worth exploring.