Health Lifestyle

Why Mainstream Reporting on the PURE Study Is Getting It Horribly Wrong

12 min read

Dr. David Katz explains how the recent reporting of the PURE Study findings is misleading.

Yes, you should still eat vegetables, fruits, and legumes – despite all the misleading claims

Imagine a new study, published, one presumes in Road & Track, or Car and Driver, purporting to show that square wheels outperform round wheels. Imagine the attendant headlines: “Everything Thought Known About Wheels Proves Wrong!” and “Wheel Guidelines Need Radical Change!”

Would such headlines, in fact, cause you to abandon everything you knew about wheels based on a lifetime of evidence and experience? Or, would you say: “that can’t possibly be true,” and just go about your business? Or, might you say, “well, wait just a darn minute…” and look further into the study, to see how such a preposterous claim could be justified in the first place?

I am guessing one of the latter options in the case of wheels. I only wish we would roll the same way when it comes to news about diet. We’ll come back to that momentarily.

First, I want to establish that my imaginary study, and its entourage of imaginary headlines, could, indeed, be feasible if there were money to be made confusing people perennially about the proper shape of tires (as there certainly is with regard to diet). How?

Well, as the headlines told you, square tires were compared to round, and square “won.” What the headlines didn’t tell you was that the square tires were made from state-of-the-art tire materials, such as vulcanized rubber. And, perhaps though square, the corners were gently rounded. The round tires were indeed round- but made out of porcelain, presumably because the study result was chosen in advance to favor the square tire industry. The porcelain tires all shattered to smithereens at the first rotation, leaving those cars stranded with no tires at all. The cars on square tires lumbered along clumsily, but they did at least move- and so, they won! The difference was statistically significant.

The above study is just the nonsense it seems. If, however, there were industries that could profit from confusion about the best shape for tires, I would not be shocked to see it. We get just such diverting nonsense about diet week after week.

The latest is the media coverage of a study called PURE (Prospective Urban and Rural Epidemiology), which ranges from mildly hyperbolic to patently absurd.

The study itself is impressive in scope, and I commend the many investigators involved for their good intentions and massive efforts. In brief, PURE was designed to look at health outcomes associated with variations in lifestyle, and in the current batch of papers diet specifically, in countries not well represented in prior work of this type, and across the range from high to very low socioeconomic status.

A total of 18 countries – with a particular focus on the Middle East, South America, Africa, and South Asia – and about 135,000 people participated. Participants were enrolled as long ago as 2003, or as recently as 2013, and were followed for about seven and a half years on average. Dietary intake was assessed with a single food-frequency questionnaire at baseline. Another dietary intake tool, 24-hour recall, was used in a sub-sample, and the correlation between the two was marginal, suggesting considerable inaccuracy in diet reporting.

Three PURE study papers were just published in the same issue of The Lancet, one reporting health outcomes (cardiovascular disease, non-cardiovascular disease, and mortality) associated with intake of vegetables, fruits, and legumes (beans, chickpeas, lentils, etc.); the second reporting on the same health outcomes with variation in the three macronutrients- carbohydrate, protein, and fat – as a percent of total calorie intake; and a third looking at variation in blood lipids and blood pressure in relation to nutrient intake.

There were two main findings that have spawned most of the mainstream media coverage and social media buzz. The first was that, while health outcomes improved and mortality declined with higher intake of vegetables, fruits, and legumes- in a multivariable analysis adjusting for other factors, that benefit “peaked” at about 3 servings per day. This has been widely interpreted to suggest that, at odds with conventional wisdom on the topic, more is not better with regard to vegetables, fruits, and beans.

The second finding garnering media attention was that across countries, the higher the intake of carbohydrate as a percent of calories, the higher the rates of disease and death; whereas the higher the percentage of calories from fat, the lower these rates.

Let’s take these in turn.

Regarding vegetable, fruit, and legume (VFL) intake:

The researchers found that those with the lowest intake (about 9000 people) of vegetables, fruits, and legumes also had the lowest intake of total calories, starch, and meat- indicating that in the many poor populations included in this study, people were simply food-deprived, and hungry.

Those with the highest intake (about 11,000 people) of VFL had nearly twice the total calorie intake of the lowest group; smoked about half as often; and were 6 times more likely to have gone to college- and were more likely to exercise (even though the poor likely did manual labor at work).

In other words, the lowest levels of VFL intake represented a fairly desperate socioeconomic status; the highest intake, more than 8 servings daily, meant privilege, and choice.

What have the crazy, hyperbolic headlines NOT been telling you? Roughly 8% of those in the lowest VFL intake group died during the study period; whereas only 3% of those in the highest VFL intake group died – despite the fact that the highest VFL intake group had a slightly higher mean age at baseline. Overall, and rather flagrantly, mortality was LOWEST in the group with the HIGHEST intake of VFL. The lowest levels of heart disease, stroke, and mortality were seen in those with the HIGHEST intake of VFL.

What, then, accounts for the strange reporting, implying that everything we’ve been told about vegetables, fruits, and beans is wrong? These benefits were “adjusted away” in multivariable models. When this method of statistical analysis was applied, the health benefit expressly attributable to VFL seemed to peak at about 3 servings per day. That, however, is fundamentally misleading – and the headlines, quite simply, were written by people who don’t have a clue what it really means.

Those people in PURE with the highest VFL intake were ALSO benefiting from less smoking, more exercise, higher education, better jobs, and quite simply- a vastly better socioeconomic existence. A multivariable model enters all of these factors to determine if a given outcome (e.g., lower death rate) can be attributed to ONE OF THEM with the exclusion of the others.  The exclusive, apparent benefit of VFL intake was, predictably, reduced when the linked benefits of a better education, better job, and better life were included in the assessment.

This no more means that VFL was failing to provide benefit in those with more education than that more education was failing to provide benefit in those eating more VFL. It only means that since those things happen together most of the time – it’s no longer possible to attribute a benefit to just one of them. Really, that’s what it means (and with all due respect to the miscellaneous headline writers untrained in the matter, I am qualified to say so).

The country-specific presentation of data showed the same gradient, with the lowest intake of VFL in the poorest regions and countries, including Bangladesh, Malaysia, Pakistan, and Zimbabwe.

Based on their multivariable models, the authors suggest that there is no clear benefit from eating more than 3 servings of VFL per day, and they propose a public health advantage in that conclusion: 3 servings a day, rather than 5, 7, or 9, represents “an approach that is likely to be much more affordable” for poor people in poor countries. Unfortunately, those same models could be used to make the same case about education: there is no clear, exclusive benefit (among those eating the most vegetables, fruits, and legumes daily) from more education- so let’s forget about college! That, too, should make things easier for the poor. I appreciate the good intentions- but the message is, simply, wrong.

What about the study of macronutrients – carbohydrate, protein, and fat?

Let’s start with dietary fat. Baseline fat intake by country ranged from a low of about 18% of calories to a high of about 30%. All of these values are considerably lower than current, average intake in the U.S. and much of Europe.

Those countries with the lowest intake of dietary fat also had the lowest intake of protein, suggesting these were people with food insecurity, having trouble obtaining adequate food intake, or dietary variety.

Saturated fat intake ranged across the countries studied from about 6% of calories to a high of about 11% of calories, again all lower than average levels in the U.S. and much of Europe, and actually very close to recommended levels. Headlines encouraging populations that already eat more saturated fat than this to add even more are not merely unjustified by anything in the study, they are egregiously irresponsible.

Unlike dietary fat, which the investigators examined in all of its various categories, carbohydrate was all “lumped” together as a single class. This produced an apparent paradox in the data: disease and death went down with more intake of vegetables, fruits, and legumes – but up with carbohydrate. What’s the paradox? Vegetables, fruits, and legumes all are comprised overwhelmingly of carbohydrate.

What explains away the apparent paradox is that vegetable, fruit, and legume intake were apparently highest in the most affluent, most highly educated study participants- while “total carbohydrate” as a percent of calories was highest in the poorest, least educated, most disadvantaged. In those cases, carbohydrate was not a variety of highly nutritious plant foods; it was almost certainly something like white rice, and little else.

The highest intake of carbohydrate as a percent of total calories was associated with lower intake of both fat and protein and was associated with higher mortality. However, much of the increase in mortality was from non-cardiovascular diseases.

So, unless you are prepared to believe that eating only white rice is the reason you are likely to be gored by a bull and bleed to death – this study doesn’t mean what the headlines say it means!

The findings actually suggest that intake of carbohydrate as a percent of total calories was highest (e.g., a diet of white rice and little else) where there was the most poverty, the least access to medical care, and the greatest risk of dying of trauma, infectious diseases, and so on.

Non-cardiovascular mortality went down as total protein intake went up across the study populations, too. Do you think this means that eating more protein prevents you from bleeding when gored by a bull – or that people in places with access to more dietary protein are less likely to be gored by a bull in the first place, and far more likely to have life-saving surgery if ever that should happen?

An alleged “surprise” in the PURE data is that higher intake of saturated fat was associated with lower mortality overall. Here, too, however, higher saturated fat intake – which occurred together with higher protein intake – was associated with much-reduced risk of non-cardiovascular death. So, does eating more saturated fat protect you from dying when run over by an ox- or does being in a place with access to more saturated fat (i.e., animal food) in the diet mean you eat the ox before he can run you over? And, that, if ever he does run into you – there’s a hospital somewhere reachable?

To be quite clear about it, there was no adjustment for, or even mention of, access to a hospital or medical care in the PURE papers.

The researchers examined the replacement of carbohydrate as a percentage of calories, with fat as a percentage of calories, but did not report variation in total calories, or the degree to which very high intake of carbohydrate as a percent of that total correlated with very low calorie intake overall, and malnutrition. Looking across the several papers, it is apparent that correlation is strong. There was also no examination of what replacing one kind of fat with another did to health outcomes, a kind of dietary variation that might have more to do with choice, and less to do with socioeconomics. This is an odd omission.

On the basis of all of the details in these published papers, the conclusion, and attendant headlines, might have been: “very poor people with barely anything to eat get sick and die more often than affluent people with access to both ample diets, and hospitals.” One certainly understands why the media did NOT choose that! It is, however, true – and entirely consistent with the data.

Also, by way of reminder: the HIGHEST levels of both total fat and saturated fat intake observed in the PURE data were still LOWER then prevailing levels in the U.S. and much of Europe, providing no basis whatsoever for headlines encouraging people already exceeding these levels to add yet more meat, butter, and cheese to their diets. Absolutely none.

As noted, the work represented by PURE, and the apparent intentions of the investigators, appear to be quite commendable. There is, however, something very odd about the timing of this observational study – independent of its rather obvious failure to address the massive impact of poverty on health outcomes.

What is odd in this case is the publication of an observational study to refute the findings of many intervention trials, including randomized controlled trials. As a rule, observational studies are used to generate hypotheses, and intervention trials – especially RCTs – are used to test those hypotheses. Observational studies come first, and only suggest associations; intervention studies come after to confirm or refute.

Personally, I have long been a proponent of observational epidemiology. I argue routinely that what we know reliably about diet, and many other things, such as putting out fires, can come from sources other than randomized trials. Generally, the most complete and purest of understanding comes when insights born of diverse sources, from intervention trials to the common experiences of a culture, are combined, and aligned. Still, it is very odd to go back to observational data once the intervention trial data have already been filed.

A number of the researchers directly involved in PURE have spent their careers, long and illustrious for some of them, nearer the beginning for others, criticizing just such observational methods. Certain investigators involved in PURE have been among the more vocal and high-profile critics, for instance, of Ancel Keys and the Seven Countries Study (SCS), impugning both on the basis of overtly false accusations about lapses and improprieties, but also on the basis of an undeniable truth: the SCS was observational epidemiology, not a randomized controlled trial.

There is a truly enormous difference, though, along with many lesser ones, between the SCS and PURE: a gap of more than half a century!

Planning for the SCS goes back some 60 years. At that time, not only did we not have RCTs to tell us much about diet and health outcomes; we did not yet even know that diet and lifestyle had any appreciable effect on the most common of such outcomes, namely heart disease. The primary question Keys and colleagues set out to address had nothing to do with any particular nutrient; it was far more fundamental. Keys was among the first to suspect that variation in diet and lifestyle produced variation in heart disease risk, that coronary disease was not simply an inevitable consequence of aging.

Perhaps it seems incredible to you now that there was ever a time we doubted a role for diet and lifestyle in coronary disease, but that simply indicates how far we have come in the last half century, how big a gap that truly is given the pace of progress. So, again, an observational study now, especially by researchers prone to propound the advantages of randomized trials, is rather odd – because we have accumulated many such randomized trials in the decades since the SCS.

We have randomized trials to show that a shift from a typical American diet to a diet richer in vegetables, fruits, whole grains, beans, lentils, nuts, and seeds- and consequently reduced in refined carbohydrate, added sugar, and saturated fat – slashes rates of type 2 diabetes in high-risk adults, far more so even than the best of medications. We have randomized trials to show that shifting from a standard, northern European diet rich in meat and dairy, to a Mediterranean diet with less of those and more vegetables, fruits, olive oil, legumes, and seafood- causes the rate of heart attack to plummet in high-risk adults. We have intervention trials to show that diets in which whole, wholesome plant foods predominate can cause coronary plaque to regress, and heart attack rates to plummet.

We also have, along with simple observations of both longevity and vitality in populations around the word that eat diets of wholesome foods, mostly plants, in various sensible and balanced combinations – an intervention study at the population level shifting diets away from meat and dairy, toward more produce, whole grains, and beans, and resulting in more than an 80% reduction in heart disease rates, and a 10 year addition to life expectancy.

We have also seen what has happened in India and China with transitions to higher intake of processed foods, meat, and dairy – and away from diets of simple plants in their native state: massively more obesity, diabetes, and chronic disease in general. We have a massive study in the entire U.S. population showing that more meat, especially processed meat, and more intake of processed foods, salt, and sugar, and less consumption of produce means more risk of premature death.

In other words, past the hype and headlines, the apparent paradoxes and puzzles, what PURE means is that: poor people with poor diets and barely enough to eat, and living in places with limited if any modern medical care – are more likely to get sick and die than people living in better circumstances. With all due respect to the researchers and none to the promulgators of massively misleading media coverage- we knew that already.

Who eats mostly plants? Two kinds of people: those who have choices, and choose plants for the many benefits; and those who have no choices at all. The former enjoy excellent health. The latter eat what they can get their hands on, struggle against the forces of poverty, and routinely die young. There is a correlation between meat intake and coronary disease, but there is also a correlation between the affluence that allows for meat intake in the first place and access to a cardiac catheterization lab. In general, those people living in places with more cardiac cath labs have more chronic disease, but avoid early death due to the advent of advanced medial care.

Before concluding this admittedly long column, one final note about the alarmingly bad timing of the PURE publications. These papers were released concurrently with the devastation in Houston, and the Gulf Coast, of Hurricane Harvey- the greatest rain event in the recorded history of the continental United States. The unprecedented rainfall is related to climate change, which in turn is monumentally influenced by global dietary choices. How appalling that the PURE findings were not merely misrepresented to the public in irresponsible reporting pertaining to human health effects, but in reporting that ignored entirely the implications of that bad dietary advice for the fate of the climate, and planet.

This week, as last, round tires are reliably better than square, assuming both are made of the same materials. This week as last, whole vegetables and fruits are reliably good for you, and for the most part, the more, the better. The benefits of that produce, however, do not preclude the benefits of an education, a job, and medical care – nor vice versa.

This week as last, summary judgment about “carbohydrate” is entirely meaningless, because that term encompasses everything from green beans to jelly beans, arugula to added sugar, and subsistence diets of white rice and little else. The vegetables and fruits, as well as the whole grains, beans, lentils, nuts and seeds in the mix, are – this week as last – good for you.

This week as last, some fats are good for you, some are bad, and some are relatively neutral; but in all cases, it depends on what you eat and what you don’t eat. This week as last, the best sources of the most beneficial dietary fats are nuts, seeds, olives, avocado, and if from animal foods- then fish and seafood.

This week as last, observational epidemiology has merit in elucidating new hypotheses worth testing in intervention trials but plays no legitimate role at all in displacing answers already predicated on just such trials.

This week as last offering up each new study out of context is like trying to make sense of an entire puzzle by examining each piece in isolation. Why we treat diet this way is the puzzle to me. As long as we do so, we can expect to make about as much progress as cars on porcelain tires.