Remembering Kevin Moore

Feinman: Recent Harvard Study On Calories Should Not Have Been Published And Exposes Sacks’ Bias

Dr. Richard Feinman exposes absurdity of the recent Sacks study

What is there not to love about Dr. Richard Feinman? This low-carb diets researcher and professor of biochemistry at the Brooklyn, NY-based SUNY Downstate Medical Center follows the evidence trail and draws his own conclusions based on what the data tells him without allowing any personal bias to get in the way. Unfortunately, not everyone in the research community holds such lofty standards for themselves.

Remember the recent Harvard study from Dr. Frank Sacks that was published in the New England Journal of Medicine in February 2009 supposedly declaring that low-fat and low-carb diets are identical and that the focus should be on calories alone? You’ll recall the media was gushing with headlines proclaiming the end to the diet debate because it’s all about the calories. Self-proclaimed health “experts” like Dr. David Katz were quoted in newspapers and on television smugly telling everyone “I told you so” that they’ve been preaching this common sense message for years and now they’ve been proven right.

Dr. Feinman says there’s only one problem with this–the study was DEAD WRONG! He said the diets labeled as “low-carb” in the Sacks study were not anything close to being a true ketogenic high-fat, low-carb diet and that the macronutrients within the four diet groups used in the study were virtually identical. Thus, it wasn’t surprising to see similar results at the conclusion of the research. To illustrate his point, Dr. Feinman made the following chart to show the conspicuous resemblance of all four diets:

Click on the graph above to ENLARGE

As you can clearly see from that graph, the four study groups in the Sacks study after adjusting for the discrepancies in their diet composition as noted by the researchers were statistically identical. Dr. Feinman concluded that this study should have never been published in any reputable medical journal in light of this shoddy research.

“In any other science, the fact that [Sacks] wanted people to take in 15% protein vs. 25% and they did not do it would be considered that the experiment failed and they would not have published it,” he wrote in an e-mail. “In fact, when he looked at the actual behavior of people in the experiment, protein did correlate with weight loss, just the opposite of what he said.”

Kudos to Dr. Feinman for adding his voice to the growing list of people who are exposing bad science when they see it. I appreciate the work he is doing at the Metabolism Society sharing genuine research that will benefit both doctors and patients with quality information to help them treat conditions like obesity, diabetes, metabolic syndrome, and other health conditions. It will be an honor to see him next month in Charleston, SC to continue working with the leaders in the low-carb research world exposing fraudulent studies when they arise.

Speaking of the Sacks study, one of the physicians on my “List of Low-Carb Doctors” blog was invited by his local ABC-TV affiliate in Denver, Colorado for a LIVE television segment to talk about the ramifications of it. His name is Dr. Jeffry Gerber and you’ll recall he was on my podcast show last year. He appeared on WMGH-TV in Denver on Tuesday sharing boldly the basic concepts of livin’ la vida low-carb. As you can see from the video clip and the news recap of this interview, Dr. Gerber did an OUTSTANDING job articulating why carbohydrate restriction is necessary for controlling weight and health. The news anchors seemed genuinely interested in and surprised by what he was sharing with them.

Send Dr. Gerber a congratulatory note on a job well done sharing about healthy low-carb living with the viewers in his area. We need more champions of the low-carb lifestyle who are able to confidently speak the truth to a world that desperately needs to hear it! WAY TO GO, Dr. Gerber!!!

3-12-09 UPDATE: Several readers have requested further explanation about what the graph Dr. Feinman created means. Here’s his response:

“In answer to your readers’ questions: I used the data from Sacks, et al. on the mean and standard deviation (spread of values) for consumption of protein (in %) to show the distribution that those values would predict (the actual values might be extracted from Figure 4 but I was giving a ball-park view). So, the x axis is the protein consumption and the y axis is the percentage of people in each group who would be expected to have the particular consumption (according to the published statistics).

What the graph shows is that all four diets had distributions that made them virtually indistinguishable from each other in terms of protein consumption: they all hit about 20% with a variation of about 5 % on either side. The paper, however, discussed the diets as if the relevant information were how the diet was set up, that is, it was intended to be (high protein) 25% or (average protein) 15%.

Now, the abstract of the article was correct and precise: weight loss remained similar in those who were assigned to a diet with 15% protein and those assigned to a diet with 25% protein. Most people feel that being assigned to a diet is not the key information: it is what you actually do that counts. It is a little like Abe Lincoln’s joke: a cat still has four feet; calling his tail a foot does not make it one.

To be fair to Sacks, this is very common in the medical literature and is a variation of “intention-to-treat” which takes the attitude that a “real world” analysis must include everybody who was assigned to the intervention regardless of what they do. There are, of course, situations where you don’t know, for example, whether or not people took a pill, and so such an approach is all you can do, but, in this case, it is misleading.

I discussed this in a paper in Nutrition and Metabolism (PDF download) where I give at least one serious example of abuse (vitamin E was judged useless because the data included people who didn’t actually take the pill. When you looked only at “compliers,” there was substantial benefit for cardiovascular disease: another good treatment is available here). The major cause is slavish addiction to statistical methods at the expense of common sense. Sensibly, the real variable in a diet experiment has to be what was actually consumed and, again, when you do that (see Figure 4 of Sacks’ paper) macronutrients appear to affect the outcome.

And, of course, the real counter-example to Sacks’ conclusion is the very large experiment with the American population. If only calories counted, there would be similar increases across the board in consumption during the epidemic of obesity and diabetes. In fact, almost all the increase in calories was due to an increase in carbohydrate. For men, the absolute amount of total fat and saturated fat went down. For women fat consumption increased only a little (percentages went down). Not to be too romantic, this result runs as a kind of basso ostinato under any discussion of macronutrients and calories.

THANK YOU for expounding upon this, Dr. Feinman! It’s always a pleasure.

  • Sue

    Gerber was good.

  • Justin Wallraven

    Can you add some labels or descriptions to the chart? I don’t understand what this chart is trying to say. What does the X and Y axis represent? What do the appreviations on the graph mean?


  • Sue

    LF-AveP = low fat, average protein
    LF-HP = low fat, high protein
    HF-AveP = high fat, average protein
    HF-HP = high fat, high protein

  • Jorge Ahumada

    Jimmy, I dont understand these graphs either. They seem to be depicting a statistical distribution, but of what? What is on the x axis? Could you post a link to the original Feinman commentary?

    Many thanks, Jorge

    Actually, there was no original commentary from Dr. Feinman, Jorge. The point of the graph is to show the actual macronutrient ratio of each of the four diets used in the Sacks study. It demonstrates the eerie similarities between the four. I will ask Dr. Feinman to explain the graph further. THANKS!


  • Justin Wallraven

    Thanks Sue. What about the X and Y axis?

  • Paula

    I love Gerber’s response, and am going to print it out and keep it handy. Of course, I love Gary Taubes too, but our most frequent criticism is that his stuff is so dense and voluminous that it’s hard to pull out what you need to challenge numerous fallacies about low-carb. This is great for having conversations with friends, giving them to read, and making comments online. It has enough medical information to be reliable, yet explains it in terms that everyone can understand. I wish every doctor could “get it” like this!

    THANKS Paula! Dr. Gerber is a really bright physician who is doing great work in the Denver, CO area. Gary Taubes is working on a more simplified version of GCBC, so stay tuned. Thank you for your feedback!


  • JayCee

    Grin…It still amaze me how any ‘expert’ can use the word CALORIE when talking about human nutrition and or energy expenditure in the human body.

    The word calorie has as much to do with the human body as the word kilometer or mile. There is no way that you can measure a calorie in the human body – digestion and combustion are two different things – the human body UTILIZES things for energy, there is no COMBUSTION…

    Its the same as comparing low-carb weight loss to the speed that a train travels from point a to b … you just cannot compare it.

  • Pellefant


    The grahps shows the normaldistribution for each diet. On y-axis its the fractions of dieters that had the corresponding in take of proteins. The with of each peak is associated with the standard deviation and the position of the peak with the average.

    One can take a note that the LF-AveP = low fat, verage protein LF-HP = low fat, high protein had less variation in protein intake around (20%) than the HF-AveP and HF-HP groups.

  • Jorge

    So the numbers 0..20..40 on the x is the % of protein in the diet? This does not say much about relative composition of carbohydrates and fat on the diet.. Just trying to make sense of this..

  • Pellefant


    Well you can atleast say that 80% is carbonhydrates or fat. Read the article it is open on the web. http://content.nejm.org/cgi/content/full/360/9/859

    The other thing is their design in average protein and high protein has failed. Since its the same for all diets, 20%!

  • dave ross

    REAL SCIENCE proves what us low carb folks know. its about time we found a real medical researcher to write about low carb. all these no-nothing media people keep coming down on low carb without any facts. lets hear more from dr. feinman and the real research that goes on in nutrition.