Diet Reporting—the Real Fake News

No one would choose to study diet as a way to understand the way humans metabolize food. Effects are delayed, often for years. Experimentation is usually impossible for ethical and practical reasons—subjects cannot be sacrificed and dissected to see the physiological effects of different food regimens. And much better methods are available to study how food is processed by the body. On the other hand, people are very interested in what they should eat. There is a huge market for ‘diet science.’

Diet Reporting Should Go on a Diet

The New York Times once had a reputation as the “paper of record,” a reliable, if left-of-center, source of information. I’m not sure it’s ever been reliable when it comes to diet.

In an article published on 26^th August headlined ‘Our Food Is Killing Too Many of Us’ (should our food be killing just the right number?), the paper recounts a list of ‘facts’ and prescriptions about how Americans—indeed, everybody—should change their diet in order to prolong their lives. The article is written by two dietary eminences: the Dean of Tufts Friedman School of Nutrition Science and Policy and an ex-Secretary of Agriculture. Yet the scientific basis for these exhortations is flimsy-to-non-existent.

The article is in a venerable tradition peculiar to health-related topics: to write in a hectoring and prescriptive tone about a subject wrapped in uncertainty; to pledge allegiance when what is required is probing, questioning or perhaps just silence.

The Times, like many other news outlets, has a long history along these lines. An early example is the treatment of a talented physiologist named Ancel Keys. He rated a TIME cover picture when the magazine reported on his massive “Seven Countries” diet study (this was back in 1961).

Keys had two main conclusions:

1. Americans eat too much. (3,000 calories/day versus Keys’ optimal of 2,300.)
2. Americans eat too much fat.

But the message the public heard was not about the amount we eat, but about the fat content.  Keys’s fat claim was later contested (sugar and carbohydrates are the opposing demon) and now (the right kind of) fat is weakly favored.

Here is a small sample of the Times‘s history of dietary proclamations:

[M]aybe…[the American medical establishment] will find that their very own dietary recommendations—eat less fat and more carbohydrates—are the cause of the rampaging epidemic of obesity in America. (‘What if It’s All Been a Big Fat Lie?,’ NYT, 7th July 2002)

In 1988, the surgeon general, C. Everett Koop, proclaimed ice cream to a be public-health menace right up there with cigarettes. (‘Diet and Fat: A Severe Case of Mistaken Consensus,’ NYT, 9th Oct 2007)

This week’s news came to us by way of a randomized controlled trial, which…is the best kind of study to determine how one thing causes another. [Actually, it’s almost the only way to be sure about cause.] One group got the “heart-healthy” low-saturated-fats diet; the other a “normal” diet.  The result: “There was…no decreased risk of death” for the heart-healthy group. (‘A Study on Fats That Doesn’t Fit the Story Line,’ NYT, 15th April 2016)

And finally:

[I]nternal sugar industry documents…suggest that five decades of research into the role of nutrition and heart disease, including many of today’s dietary recommendations, may have been largely shaped by the sugar industry….‘They were able to derail the discussion about sugar for decades,’ said Stanton Glantz, a professor of medicine at U.C.S.F. and an author of the JAMA Internal Medicine paper. (‘How the Sugar Industry Shifted Blame to Fat,’ NYT, 12th Sept. 2016)

Notice the trend. First the papers buried a claim that has never been contested—that we eat too much—with more newsworthy claims about dietary details that are still being debated: the virtues and sins of types of fat and carbohydrate. Then a debate—fat versus carbs—follows, leaving the public confused. And finally—at last a good story!—it all turns out to be a plot by “Big Sugar.”

With this kind of history, one might expect responsible editors and headline writers to be a bit cautious when reporting the latest diet study. But no, “Our food is killing us” or at least “too many” of us, says the Times. The article, which was presented as an authoritative proclamation on the rights and wrongs of diet, is in fact close to nonsense—or at least a new religion. It is not science. Let’s look at a few of these grandiose claims:

Americans are sick—much sicker than many realize.” It’s original sin and color-blind racism all over again! You feel fine, but no, you’re sick. How do they know? Well, you’re fat and have high blood pressure, and maybe your triglycerides are too high. Why are these things bad? Well, high BP is “correlated with” heart attacks and stroke. I see. Does high BP cause these events? Well, actually, we don’t really know. In other words, most of these “proxies” for health have been establish by correlation. The real, causal chain is in most cases completely unknown. But you are really sick…

How about this claim?

Poor diet is the leading cause of mortality in the United States, causing more than half a million deaths per year. Just 10 dietary factors are estimated to cause nearly 1,000 deaths every day from heart disease, stroke and diabetes alone…

These figures are necessarily fantasy, because there is simply no way they could ever be measured. Can we really pinpoint deaths due to “residential radon and lead exposure”? And, once again, causes are inferred from data that are entirely correlational. Death statistics can be relied on; the causes of death, especially remote causes like diet, as opposed to proximal causes like heart failure, cannot.

The article continues:

These conditions are dizzyingly expensive. Cardiovascular disease costs $351 billion annually in health care spending and lost productivity, while diabetes costs $327 billion annually. The total economic cost of obesity is estimated at $1.72 trillion per year, or 9.3 percent of gross domestic product.

These numbers make about as much sense as GDP predictions 10 years in the future. How were the costs of, for example, diabetes, computed? These are cross-sectional—annual —costs; how do they relate to lifetime costs versus lifetime health costs for the non-diabetic population? This is surely the statistic that matters for public health. Were they just medical costs? Was diabetes the only condition present? (Probably not, since obesity and diabetes are correlated.) Did they exclude the medical costs that would have been incurred if the person had died from some other cause (we all die of something, after all, and dying is expensive)? Well, no, of course not: it was just spending more or less related to diabetes.

These figures come from a 2018 JAMA report from something called The US Burden of Disease Collaborators, an international group consisting of hundreds of people, all listed at the end of the article, funded by the Bill and Melinda Gates Foundation and based on data crunched by a “25,000 CPU supercomputer.” With all that power, the report must be right, yes?

Well, actually, no. With 333 potential causes and 84 risk factors and a vast dataset compiled from dozens of separate surveys, there will be many correlations: fat correlated with diabetes, sugar with stroke, etc. When they can be tested experimentally—and very few can—almost none are likely to prove causal (see below).

The Truth

Here are some messages, from another JAMA article, and one from the Times itself, which might have prevented the Times editors from publishing their hortatory nonsense last month:

[M]uch of the public often consider epidemiologic associations of nutritional factors to represent causal effects that can inform public health policy and guidelines. However, the emerging picture of nutritional epidemiology is difficult to reconcile with good scientific principles. The field needs radical reform…Almost all nutritional variables are correlated with one another; thus, if one variable is causally related to health outcomes, many other variables will also yield significant associations in large enough data sets…Nutritional research may have adversely affected the public perception of science. (JAMA, 11^th Sept. 2018, from an article entitled ‘The Challenge of Reforming Nutritional Epidemiologic Research’ by John Ioannidis, the man who first pointed out the replicability crisis that social and biomedical science are now struggling with.)

Richard Peto, professor of medical statistics and epidemiology at Oxford University, phrases the nature of the conflict this way: “Epidemiology is so beautiful and provides such an important perspective on human life and death, but an incredible amount of rubbish is published,” by which he means the results of observational studies that appear daily in the news media and often become the basis of public-health recommendations…

In January 2001, the…co-editors of The International Journal of Epidemiology…noted that those few times that a randomized trial had been financed to test a hypothesis supported by results from these large observational studies, the hypothesis either failed the test or, at the very least, the test failed to confirm the hypothesis…” (From an article entitled ‘Do We Really Know What Makes Us Healthy?’ by science journalist Gary Taubes, in the New York Times Magazine, 16^th Sept. 2007. All the NYT editors needed to do before publishing their latest piece of diet reporting was read their own coverage of the issue!)

Never mind fat versus carbs. Ancel Keys’s uncontested obesity cure is eat less! The recommendation as far as diet journalism is concerned is much the same: Don’t swallow everything!