Prestigious Science Journals Struggle to Reach Even Average Reliability by Björn Brembs.
Abstract:
In which journal a scientist publishes is considered one of the most crucial factors determining their career. The underlying common assumption is that only the best scientists manage to publish in a highly selective tier of the most prestigious journals. However, data from several lines of evidence suggest that the methodological quality of scientific experiments does not increase with increasing rank of the journal. On the contrary, an accumulating body of evidence suggests the inverse: methodological quality and, consequently, reliability of published research works in several fields may be decreasing with increasing journal rank. The data supporting these conclusions circumvent confounding factors such as increased readership and scrutiny for these journals, focusing instead on quantifiable indicators of methodological soundness in the published literature, relying on, in part, semi-automated data extraction from often thousands of publications at a time. With the accumulating evidence over the last decade grew the realization that the very existence of scholarly journals, due to their inherent hierarchy, constitutes one of the major threats to publicly funded science: hiring, promoting and funding scientists who publish unreliable science eventually erodes public trust in science.
Facts, even “scientific facts,” should be questioned, tested and never blindly accepted.
Knowing a report appears in Nature, or Science, or (zine of your choice), helps you find it. Beyond that, you have to read and evaluate the publication to credit it with more than a place of publication.
Reading beyond abstracts or click-bait headlines, checking footnotes or procedures, do those things very often and you will be in danger of becoming a critical reader. Careful!
