Tedious publishing: Preregistration can help

Imagine that an architect were tasked with building a house, came up with a design, had a team of builders construct the house and then asked the prospective owner whether anything needed to be changed. The prospective owner tells the architect that he doesn’t like a part of the house and would rather it were done differently. To please the owner, the architect knocks down that section of the house, rebuilds as directed and then repeats the process until the prospective owner is satisfied.

To any sane person, this situation would seem completely ridiculous. It would be stupid to go through all the effort of constructing a building without first confirming whether everything was acceptable at the planning stage. Strangely enough, this is not far off from the common practice in science. We design and perform experiments, collate and narrate the results and then send our write-ups to journal editors, who in turn distribute the manuscript among a coterie of our peers. These peer reviewers supposedly check the manuscript for publication-worthiness, advise us of errors in execution or interpretation, and recommend various changes and additions. Frequently, this turns out to be a prolonged exercise that incurs significant expense, but does not add much value to the central theme of the manuscript. Without a doubt, the scientific process can benefit from well-performed peer review. However, extant peer review processes are more wasteful than beneficial as in the analogy above.

Advertisement

Science: Behind the scenes

I’ve collected a few weird stories from the world of science that are interesting. They’re not science stories about discoveries and research but they’re science stories about what goes on behind the scenes.

Spam and mailing lists

Let’s start with an amusing tale. We’ve all got spam before and scientists are no exception. Fed up with the constant spam from one particular journal, Dr. Peter Vamplew submitted a fake article, originally written by David Mazières and Eddie Kohler, that expressed his frustration. The article was 10 pages long and merely consisted of the sentence, “Get me off your fucking mailing list” repeated throughout.

Science: DIY, peer review and problems

DIY science seems quite interesting, especially if you have an interest in science but are following a different career path. There are various ways to get by without the usual instruments, although I’m not sure how successful they are, but this guide to turning your smartphone into a microscope seems pretty good. I’ve actually got a busted laser pointer (the battery compartment doesn’t shut) so I might try this at some point.

The following is a story I heard about at the Southern African Society of Human Genetics conference. According to a news article published in Science magazine a series of obviously fake papers were created and sent to 304 open access journals.

By the time Science went to press, 157 of the journals had accepted the paper and 98 had rejected it. Of the remaining 49 journals, 29 seem to be derelict: websites abandoned by their creators. Editors from the other 20 had e-mailed the fictitious corresponding authors stating that the paper was still under review; those, too, are excluded from this analysis.

It’s a worrying result but a useful one. For example, it shows that peer review works at PLoS One and BioMed Central. In addition it told me that there is poor peer review at Dove Press and the OMICS publishing group, which is worrying because I had been getting Dove Press alerts (though never read their articles) and had heard senior scientists mention OMICS as a possible place to publish. Needless to say I cancelled my subscription to Dove Press alerts and will advise against publishing with OMICS.

The main problem with the experiment, and it is at least mentioned in the article itself, is that there’s no control group. It describes this as an open access issue but doesn’t give us the data to say that it doesn’t happen in subscription journals. This issue has been raised elsewhere, like here and here, the latter with further criticisms.

Lastly, we have a sad story about the many problems currently plaguing science. While I would suggest reading at that link and the two articles it references, if you only have time for one I would recommend this one. There are many scary things included, lack of replication of studies, problems with peer review and poor use of statistical significance to name a few. Many I’d already heard of but it’s distressing to see it all laid out in one place. But it’s only if we know about it that we can start addressing it.