Mark Hanna’s greatest passion is debunking bad science. Here, he looks at the New Zealand Beverage Council’s recent media campaign claiming soft drinks aren’t a big factor in obesity.

The New Zealand Beverage Council’s president was interviewed on Morning Report today about a curious, conveniently self-serving claim he’d made in a recent media release:

“Olly Munro, President of the New Zealand Beverage Council says the data clearly shows that the buying and consumption patterns of Kiwi consumers indicates that soft drinks – being targeted for a soda tax – are not the significant contributor to the obesity problem they are made out to be.”

The survey Munro’s talking about, which the Council commissioned from the survey company Nielsen, looked at Kiwis’ consumption habits; things like how often we drank soft drinks compared to products like bottled water. It didn’t actually look at how big a factor soft drinks are in obesity. That’s a big problem for his claim it shows soft drinks aren’t a big factor in obesity.

Luckily for us, that seems to have been Guyon Espiner’s first thought as well. The Morning Report host brought up this criticism early in his interview with Munro. “Yeah it doesn’t though, as you claim, prove that sugary drinks aren’t a factor in obesity, does it?” he noted, dryly.

So how did the Council conclude soft drink consumption isn’t contributing that much to obesity from a study that didn’t really look at how soft drink consumption is contributing to obesity? Was it calculated misdirection, or honest incompetence?

When I tweeted about the interview, the Council replied with a link to an infographic they’d published.

Of course, an infographic  on their own website isn’t exactly on par with peer-reviewed research, but it seemed like they were at least trying to back up their claim. Perhaps the infographic contains some information about soft drinks not contributing significantly to obesity?

Well… no, it doesn’t. There are lots of numbers about how often people consume different drinks, and how much sugar the whole population gets from various sources. But it doesn’t mention obesity at all.

So I asked the Council to help me find which specific part of the infographic backed up their claim about how much of an effect soft drinks have on obesity, and they answered. Only, their answer didn’t make sense.

When I pointed out that the frequency Kiwis consume soft drinks compared to other drinks, on average, is not the same as how much they contribute to obesity, they agreed.

It’s all very confusing.

Even the information within this infographic deserves some scepticism. Like many infographics, they use “simplifying the data” as an excuse to provide vague and ambiguous statistics like “Kiwis drink twice as much alcohol vs soft drinks on a weekly basis”. Does this mean twice as many of us drink alcohol on a weekly basis as drink soft drinks weekly? Or perhaps it means we drink twice as much alcohol by volume as we do soft drinks?

The general point of the infographic, in line with the claim in their press release, seems to be that soft drinks aren’t that bad because we don’t drink them that much. Stats like “Kiwis choose to drink water a third of the time, while soft drinks are chosen less than 4% of the time” and “32% of Kiwis never drink soft drinks” are thrown in to back that impression.

But we should look harder at claims like “32% of Kiwis never drink soft drinks”. Does that mean all these statistics about how few soft drinks we consume are including data from a large proportion of people who never drink them?

This kind of thing is a big problem with population-wide statistics like these: they smooth everything out so that people who aren’t affected by a problem make the big picture look better than it is for those who are.

Further down, the infographic points out nonchalantly that 21% of Kiwis’ added sugar intake comes from soft drinks. But really that figure would be 0% for those Kiwis who never drink them, and 31% for those who do.

Something else to always keep in mind when hearing about surveys like this is that, if the industry group that paid for it wasn’t happy with some or all of the results, it is very easy for them to sweep those under the rug. These surveys simply are not about informing you, they’re about presenting the picture the industry wants you to see.