Journalists Called Out for Bad Reporting on Consumption Data

Being a journalist is not an easy job; it demands fast paced and high volume production. For those “wonk” journalists tasked with analyzing data-heavy reports for laymen readers, the task is even more difficult. A new post from Forbes’s Trevor Butterworth scrutinizes some recent articles from The Washington Post’s Wonkblog, highlighting how some journos aren’t exactly rising to the challenge. And the consequences go beyond a misinformed public.

On September 25, Wonkblog posted an article claiming that 24 million American adults are consuming a shocking average of 74 drinks per week. Butterworth delves into the source of those data and points out the numbers aren’t exactly accurate. The data were pulled from a 2007 report which used information from a survey of Americans’ consumption during 2001-2002. Of course, self-reporting surveys are notoriously flawed. Participants misremember what they ate, miscalculate how much, or outright lie about their consumption habits. Butterworth states that if people really consumed what they tell researchers in self-reporting surveys, “life for almost two thirds of Americans would be biologically implausible.”

Cook, understanding these flaws, attempted to correct for underreporting by multiplying the reported number of drinks by 1.97 which, as Butterworth noted, “requires us to believe that every drinker misremembered by a factor of almost two. This might not much of a stretch for moderate drinkers; but did everyone who drank, say, four or eight drinks per week systematically forget that they actually had eight or sixteen? That seems like a stretch.”

Furthermore, the study also requires us to believe that those reporting no consumption didn’t drink a drop and even more significantly problematic that those under 18 years old—who were not included in Cook’s study—never consumed alcohol. Plenty of data demonstrates that even though they aren’t legally supposed to drink, a fifth of 12th graders are consuming alcohol. Additionally, Cook’s study, which was trying to account for a discrepancy between how much people said they consumed versus how much alcohol was sold in America, did not account for waste. As Butterworth points out, “…waste is a huge issue with food, with estimates running from 30-40 percent of calories produced; we do not know how much alcohol is, if you’ll forgive the pun, wasted.” Lastly, he notes that other studies looking at consumption patterns “[a]ll converge on a similar proportion; none come remotely close to Cook’s estimate; none are mentioned in Wonkblog.”

Unfortunately, this isn’t the first case data-flubbing at Wonkblog. Butterworth also points to a post boldly titled, “Why the U.S. should start taxing soda like cigarettes and alcohol,” in which Roberto Ferdman claims that “America’s love affair with soda has occurred alongside an eerily similar climb in the country’s sugar intake.” The post points to per capita increase in sugar along with the rise in obesity. But in actually looking at the data, Butterworth finds:

If you put all this data together, you can see the following trends: In 1970, total calorific sweetener consumption per person was 20.8 teaspoons a day; in 1980, it was 21 teaspoons a day; in 1990, it was 23.1 teaspoons a day, in 2000, it was 26 teaspoons; in 2010, in was 22.9; and in 2013, it was 22.3…

And he asks, given this data, “[w]as the obesity crisis really born in the late 1980s and 1990s from a gradual aggregate increase of about five teaspoons of calorific sweetener a day (80 calories)? And if so, why have we not seen a corresponding decline in weight over the past decade given that we are returning to levels of consumption not seen since the 1970s?” That question is one that should have been raised at Wonkblog, but it wasn’t, because the article is an opinion piece disguised as data analysis. “The concept of a national sugary drinks tax isn't even all that novel considering the widely accepted and long-held taxes levied on tobacco and alcohol—the two are taxed at both the federal and state level to compensate for their respective social costs. Diabetes is the seventh leading cause of death in this country. It's not unreasonable to view sugary drinks and sodas, which are as inessential as cigarettes and alcohol, in the same light,” Ferdman concludes.

Thankfully, there are reporters out there reminding us that we should always be skeptical of news stories—even those from so-called wonks.