Ever seen a headline about a crazy new scientific discovery and thought “pfft, the fuck outta here… but wait, really?
No. It’s almost never “really.”
Today’s moment of science… read between the headlines.
Messy science gets into your newsfeed a variety of ways. It’s been years since the chocolate weight loss hoax, and I still haven’t decided if it was the cause of more insight or annoyance.
John Bohannon, a scientist with a bit of a sense of humor (I can’t say I’m not a fan), embarked on a small scale dietary study with fifteen participants. One group was put on a low carb diet, another had the same but included a small piece of chocolate daily. The third group was a control group, instructed to continue their current diet. The researchers tracked eighteen variables over three weeks, and then the P-hacking began. P-hacking is a term that roughly means they tortured the data until it cried uncle.
With eighteen variables and just five people per group, the study was designed to create some fast and dirty statistically significant data. After a lot of creative mathematics, they found the group that ate chocolate lost just enough more weight than the other groups to write an eye-catching headline.
They dropped the study in a pay-for-play journal, where it was way too easy to publish.
The plan worked a little too well and the story went viral. Amongst others, the Irish Examiner, Times of India, Huffpo, Shape, and Prevention all picked up on the story. Unfortunately, they had to explain to disappointed chocolate lovers that no, the results of five people over three weeks in a study designed to get shallow results did not mean that chocolate causes weight loss.
I still occasionally have to explain to people that the story was fake.
What about when nobody corrects the story because it wasn’t a hoax, but the media is painfully far removed from the research?
Watching scientific results move from the scientist to the university’s press office and then out to the public often seems like a really bad game of telephone. Inevitably, someone has to explain that no, they did not say to put the log in the vegetable quiche (I said put the dog on the extendable leash, I swear). Or more ludicrously, debunk headlines that a glass of red wine is the same as an hour of exercise.
If that had been true, marathon training would have been fucking rad.
Jason Dyck, a researcher at the university of Alberta, looked into the effects of resveratrol, a component of things like grapes (and red wine) in order to maximize exercise benefits for people with limited mobility. It was found to have benefits.
In mice.
In a supplement.
In a quantity of resveratrol that you’d find in 100-1000 bottles of wine.
“A lot of times when you work away in your lab you hope that people read your studies and that it makes an impact. When you see all your hard work distilled down to one phrase that isn’t correct it’s a little disheartening,” Dyck said in an interview.
Then there are the times when people don’t know how correlations work, and shit gets messy. Everything you eat, breathe, drink, and lubricate can kill you if headlines based on correlation are to be believed.
So we’re going to have to talk about Nicolas Cage. Because he’s been up to some shit.
If we treated Nicolas Cage the way we treated anything else when we get nervous with a correlation, he’d be in a level five biohazard containment unit for life. The number of films Nicolas Cage is in during a year has an oddly strong correlation with deaths by drowning in a swimming pool (see attached photo). The number of his films also has a strong correlation with men slipping or tripping to their deaths.
He’s a wizard. I’ve done my research.
A correlation isn’t a bad place to start researching in science. You see two things happening at the same time, it makes sense to investigate if there’s a causative link. Bad reporting frequently muddies the waters between correlation and causation. Sometimes perfectly innocent medications, chemicals, and foods have been made suspect for our health woes when they were just passing through.
This all might make you ask some questions about your news ecosystem and how you interact with it. Do you regularly check out the study being discussed when you read a pop article about science? How carefully should a journalism outlet preserve complex science in a concise headline? How do you safeguard yourself from bad science in the media… and Nicolas Cage?
This has been your daily Moment of Science suggesting that if you see someone downing chocolate and wine, don’t hate for finding a scientific approach to weight loss and exercise.
To get the column sent straight to your inbox, for bonus content, and help support my efforts to fact check the internet, head to patreon.com/scibabe.
Image source: tylervigen.com- go check out the website for loads more spurious correlations.
Having watched a few Nicolas Cage stinkers (Wicker Man, anyone?) I’ve definitely contemplated drowning myself afterwards. So, maybe there is something to that correlation?