Bad science reporting

Science reporting can be tricky but if your job is to report on such topics then you should do it responsibly. A recent article on Yahoo!, written by Elise Solé, shows a number of bad reporting habits that should be discouraged, starting with the sensationalist headline.

Why Teen Girls Should Avoid Soda

Drinking sugary drinks is bad for you and everyone should try to cut down. I essentially stopped drinking sugary drinks a few years ago for the sake of my teeth but there are many benefits to cutting them completely or merely reducing consumption. However, that’s not what this article is about. Compare it to a headline from another news article on the same subject (I’ll tell you why I chose this one later).

Soda Habit May Prompt Early Puberty in Girls, Study Suggests

Now the headline reflects the content of the article. A study was recently published that found an association between drinking sugar-sweetened beverages and earlier puberty. This may have health consequences, which would support the need to reduce consumption of sugary drinks, but it’s not the final word.

As we continue reading the Yahoo! article we find this sentence:

Now, researchers have added another health hazard to the mix: Girls who drink lots of soda enter puberty earlier than girls who don’t.

It comes with a link! You’d expect that clicking that link would take you to the study but it doesn’t. In reality it just takes you to the US News article I linked above. That’s a bad sign.

The US News story is a convenient place to link to but, ideally, you should link back to the primary source. In this case, the link is both misleading and lazy. The primary source is right here! It’s online and free to access (unlike some articles). This suggests to me that Solé hasn’t actually read the original article.

Study title

Her statement is also false; the paper never claimed to show that drinking lots of soda cause a girl to enter puberty earlier. The study authors know this and state, in their conclusion, “Our findings suggest that frequent consumption of SSBs may be associated with earlier menarche.” This is the oft-repeated “correlation is not causation“. Seeing an association does not tell us whether drinking soda causes early puberty, if early puberty causes someone to drink more soda or if a third factor causes both early puberty and increased consumption of soda.

Even if Solé hadn’t read the original paper, she should have known this as the article she links to includes a quote from the study’s senior author saying the same thing.

Michels cautioned that this study cannot prove directly that sugar-sweetened drinks are the cause of earlier menstruation. “We are showing an association,” she said. “We can only do some guesswork on the mechanisms.”

We can also see, like in broken telephone, a decrease in the accuracy of basic statements. The original study states they studied 5583 girls. The US News article simplifies that to “nearly 5,600.” In the Yahoo! article, Solé mangles the approximation to say they “studied 5,600 girls.” Nearly 5600. About 5600. Approximately 5600. But not 5600. It’s a small difference but even small differences matter. Surely you would feel differently if you the doctor told you that you had a canker versus telling you that you had a cancer?

There are some other odd points like:

According to the Centers for Disease Control (CDC), teens and young adults consume more sugary drinks than any other age group — that may explain why the study didn’t include girls who don’t drink soda.

The logic there doesn’t make sense. Just because teens, as a group, drink more soda than any other group does not mean that the proportion of teens who drink soda differs from other groups. For example, if 4/10 adults drink soda and each one of those four drinks two litres a week then adults, as a group, drink 0,8 litres per week per person. If 4/10 teens drink soda and each one of those four drinks four litres a week then teens, as a group, drink 1,6 litres per week per person. Teens are drinking more soda than adults but the proportion of teens that are drinking soda is the same as the proportion of adults that are drinking soda.

I’m also not entirely sure what she means when she says it might explain why they didn’t include girls who don’t drink soda. Their methods were fairly clear about how they chose study participants and there was no exclusion due to soda consumption. In fact, as the low consumption group is fewer than one glass per week, there may be girls that do not drink any sugar sweetened drinks in the study. I don’t recall seeing any mention of the range of soda consumption.

The other article has its own strange inaccuracies. Firstly, the date at the top of the page says 28 Jan, which I assume was when it was published. The date at the start of the text says 27 January, I guess when it was written. But, when writing about the study, the author says, “The report was published online Jan. 28 in the journal Human Reproduction.” If the article was written of the 27th then it’s like saying, “I went on holiday next week.” Alternatively, if they wrote for the publication date, why include the 27 January in the beginning? To further complicate matters the top of the study pdf clearly states, “Hum. Reprod. Advance Access published January 27, 2015.” US News really needs to get their dates straight.

It’s actually worse than the Yahoo! article in one important aspect. Throughout the entire US News article there isn’t a single link! There is one after the story but the page it links to is not even mentioned in the text. Now that was perfectly acceptable when news was printed but, to be blunt, if you have an article online without a single link in it then you have not done your job properly.

So here are some guidelines for writing about science, or anything really:
Don’t oversell it! Explaining possible implications is good, blowing it out of proportion is bad.
Link to your sources! And, wherever possible, make the link to the primary source. There’s no reason not to include a link online and it’s an endless source of frustration when someone writes about something interesting and you have to waste time trying to find it.
Accuracy is important! Tell the truth, perhaps only the best bits of the truth and nothing but the truth.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s