Dr. Richard Feinman exposes absurdity of the recent Sacks study
What is there not to love about Dr. Richard Feinman? This low-carb diets researcher and professor of biochemistry at the Brooklyn, NY-based SUNY Downstate Medical Center follows the evidence trail and draws his own conclusions based on what the data tells him without allowing any personal bias to get in the way. Unfortunately, not everyone in the research community holds such lofty standards for themselves.
Remember the recent Harvard study from Dr. Frank Sacks that was published in the New England Journal of Medicine in February 2009 supposedly declaring that low-fat and low-carb diets are identical and that the focus should be on calories alone? You’ll recall the media was gushing with headlines proclaiming the end to the diet debate because it’s all about the calories. Self-proclaimed health “experts” like Dr. David Katz were quoted in newspapers and on television smugly telling everyone “I told you so” that they’ve been preaching this common sense message for years and now they’ve been proven right.
Dr. Feinman says there’s only one problem with this–the study was DEAD WRONG! He said the diets labeled as “low-carb” in the Sacks study were not anything close to being a true ketogenic high-fat, low-carb diet and that the macronutrients within the four diet groups used in the study were virtually identical. Thus, it wasn’t surprising to see similar results at the conclusion of the research. To illustrate his point, Dr. Feinman made the following chart to show the conspicuous resemblance of all four diets:
Click on the graph above to ENLARGE
As you can clearly see from that graph, the four study groups in the Sacks study after adjusting for the discrepancies in their diet composition as noted by the researchers were statistically identical. Dr. Feinman concluded that this study should have never been published in any reputable medical journal in light of this shoddy research.
“In any other science, the fact that [Sacks] wanted people to take in 15% protein vs. 25% and they did not do it would be considered that the experiment failed and they would not have published it,” he wrote in an e-mail. “In fact, when he looked at the actual behavior of people in the experiment, protein did correlate with weight loss, just the opposite of what he said.”
Kudos to Dr. Feinman for adding his voice to the growing list of people who are exposing bad science when they see it. I appreciate the work he is doing at the Metabolism Society sharing genuine research that will benefit both doctors and patients with quality information to help them treat conditions like obesity, diabetes, metabolic syndrome, and other health conditions. It will be an honor to see him next month in Charleston, SC to continue working with the leaders in the low-carb research world exposing fraudulent studies when they arise.
Speaking of the Sacks study, one of the physicians on my “List of Low-Carb Doctors” blog was invited by his local ABC-TV affiliate in Denver, Colorado for a LIVE television segment to talk about the ramifications of it. His name is Dr. Jeffry Gerber and you’ll recall he was on my podcast show last year. He appeared on WMGH-TV in Denver on Tuesday sharing boldly the basic concepts of livin’ la vida low-carb. As you can see from the video clip and the news recap of this interview, Dr. Gerber did an OUTSTANDING job articulating why carbohydrate restriction is necessary for controlling weight and health. The news anchors seemed genuinely interested in and surprised by what he was sharing with them.
Send Dr. Gerber a congratulatory note on a job well done sharing about healthy low-carb living with the viewers in his area. We need more champions of the low-carb lifestyle who are able to confidently speak the truth to a world that desperately needs to hear it! WAY TO GO, Dr. Gerber!!!
3-12-09 UPDATE: Several readers have requested further explanation about what the graph Dr. Feinman created means. Here’s his response:
“In answer to your readers’ questions: I used the data from Sacks, et al. on the mean and standard deviation (spread of values) for consumption of protein (in %) to show the distribution that those values would predict (the actual values might be extracted from Figure 4 but I was giving a ball-park view). So, the x axis is the protein consumption and the y axis is the percentage of people in each group who would be expected to have the particular consumption (according to the published statistics).
What the graph shows is that all four diets had distributions that made them virtually indistinguishable from each other in terms of protein consumption: they all hit about 20% with a variation of about 5 % on either side. The paper, however, discussed the diets as if the relevant information were how the diet was set up, that is, it was intended to be (high protein) 25% or (average protein) 15%.
Now, the abstract of the article was correct and precise: weight loss remained similar in those who were assigned to a diet with 15% protein and those assigned to a diet with 25% protein. Most people feel that being assigned to a diet is not the key information: it is what you actually do that counts. It is a little like Abe Lincoln’s joke: a cat still has four feet; calling his tail a foot does not make it one.
To be fair to Sacks, this is very common in the medical literature and is a variation of “intention-to-treat” which takes the attitude that a “real world” analysis must include everybody who was assigned to the intervention regardless of what they do. There are, of course, situations where you don’t know, for example, whether or not people took a pill, and so such an approach is all you can do, but, in this case, it is misleading.
I discussed this in a paper in Nutrition and Metabolism (PDF download) where I give at least one serious example of abuse (vitamin E was judged useless because the data included people who didn’t actually take the pill. When you looked only at “compliers,” there was substantial benefit for cardiovascular disease: another good treatment is available itt.htm>here). The major cause is slavish addiction to statistical methods at the expense of common sense. Sensibly, the real variable in a diet experiment has to be what was actually consumed and, again, when you do that (see Figure 4 of Sacks’ paper) macronutrients appear to affect the outcome.
And, of course, the real counter-example to Sacks’ conclusion is the very large experiment with the American population. If only calories counted, there would be similar increases across the board in consumption during the epidemic of obesity and diabetes. In fact, almost all the increase in calories was due to an increase in carbohydrate. For men, the absolute amount of total fat and saturated fat went down. For women fat consumption increased only a little (percentages went down). Not to be too romantic, this result runs as a kind of basso ostinato under any discussion of macronutrients and calories.
THANK YOU for expounding upon this, Dr. Feinman! It’s always a pleasure.
