How to recognise bad papers?

We first discussed what a good paper is to us. Most of us agreed that a good one is one that is useful to the reader in a reusable way: either because it gives ideas to be reused in later projects, references or examples to be used in writing papers and of course, methods and datasets that can be re-used. On the other hand, we considered as bad papers, papers that cannot be reused in the context highlighted above: they are not reproducible (no clear methods or results, missing or unavailable code or data, etc), they are not clear, can lack context and missed their audience, they are overselling results or methods. Here is an infographic by Andy Brunning summarising some of these points.

What are questionable practices in eco evo? Is it a big problem?

We then discussed what are question practices in eco-evo and whether they are a problem based on this paper. Among the questionable practices, we discussed a lot HARKing (post-hoc hypothesis), switching analyses, adding/removing data and unreported model/results. Interestingly we agreed that all these practices were actually fairly common in our work: this is because all these “questionable” practices are used as sensitivity analysis or/and because some of our work is based on null modelling. This led us to discuss the intention about these questionable practices: are they used to make the results more robust and the paper more useable to the readers (making a good paper), or are they used to oversell a project or cutting corners (making a bad paper). This was an interesting aspect of questionable practices where the practices ought to be questioned in the context of the authors’ intentions.

How to recognise or find good papers?

One main criteria is to choose where to read and eventually where to publish your paper. There are official University of Sheffield guidelines that can help. One main source of bad papers are predatory journals and publishers. To avoid them here is some help to identify trusted publishers for your research.

Another useful resource is Retraction Watch, a blog that reports on retractions of scientific papers and on related topics. The blog is a US non-profit organisation. It was created to encourage continuation of peer-review after the publication and to increase the transparency of the retraction process. They observed that retractions of papers generally are not announced, that the reasons for retractions are not publicised, and that other researchers or the public who are unaware of the retraction may make decisions based on invalid results. Keep in mind that the retraction process is very long (but there is a hope it will change soon!). It sometimes takes a few years from decision to retraction (Example). Finally you can check Pubpeer (blog, chrome plug-in) regularly to see/flag any concerns about any particular paper (post-publication peer review).

List of different fraud types and examples:

• Data (example: Jonathan Pruitt)
• Analysis (p-hacking…)
• Image misconduct (see work from Elisabeth Bik highlight examples and presentation by Slides by Kaveh Bazargan)
• Plagiarism (software designed to change words in such a way that text remains undetectable as plagiarized during a plagiarism check - example below by Slides by Kaveh Bazargan slides 28-29)