Science and Technology Committee (15th Dec 2021) - Reproducibility and research integrity

Discussion re: publication bias between Dr Greg Clark MP (Chair of Science & Technology Select Committee) and Dr Elizabeth Moylan (Publisher, Research Integrity and Publishing Ethics, Wiley).

Greg Clark: Perhaps I can turn to Dr Moylan. In terms of this question about involvement earlier than at the final point of publication, one of the suggestions that was made, is that journals, perhaps for reasons that are understandable, are more likely to publish results that are interesting. Which means that results that have been very faithfully and rigorously researched, that confirm an existing hypothesis, or fail to support a new one, simply don't see the light of day, and that gives a bias in what's published. In that an inevitable feature of a journal who - there's a range of popularity and sales associated with different journals I realise - but do they share a common denominator that the readers want to see what's interesting and new, rather than what may turn out not to be?

Elizabeth Moylan: I don't think that's a problem of the journal. I think that's a problem of how the researchers perceive things. And often they will want to show the positive stories and not necessarily report on the "oh, that was a bit confirmatory", but journals would welcome that.

Greg Clark: You're as likely to report, say, a paper that is confirmatory, as one that is new, as it were?

Elizabeth Moylan: Yes, because it validates that previous study and it reproduces that previous study. There's a variation across journals, and there's, you know, journals which are absolutely geared up to publish sound science, whatever the outcome, so we would welcome that, and we would share that. I think it's a problem of researchers getting round to sharing that it's the sort of file drawer problem that's been referred to, but we would welcome publication of those results.

Greg Clark: You have a portfolio that deals with research integrity and publishing ethics. Have you got any statistics to bear out that interesting observation that you made, that you're as likely to publish something that is confirmatory, as new and surprising?

Elizabeth Moylan: So, we would have to check what each journal was publishing...so we publish over 16,000 journals, so it would take a lot of effort to show that in a particular...

Greg Clark: We are talking about research... I mean that's...when one has a large number of observations, one has methods for doing that - that's not a trivial question.

Elizabeth Moylan: I think what you're asking is: within the journals that we publish, how many of those articles are publishing confirmatory sound science results, as opposed to exciting results - is that the question you're asking?

Greg Clark: Well, you've made a very interesting observation...what we heard from the previous panel was that there was much more interest in journals on publishing research results that were bold in what they found and surprising, to some extent. You told the committee that that is not the case and that you're as likely to publish confirmatory papers as non-confirmatory, so I'm asking for the evidence behind your very interesting assertion.

Elizabeth Moylan: It's to do with the journals and their scope - so if they are willing to publish sound science articles in that area then they will...it's...

Greg Clark: I'm not asking you about their intention, I'm asking about the evidence, whether you have evidence to back up the assertion that you've made?

Elizabeth Moylan: Yes, I do believe we will have that, and we can share that with the committee. I'll have to talk to my colleagues to show that we do publish sound science results...but it has to come from the researchers - so it's not the publishers saying that we won't publish that.

Source: bit.ly/moylanpublicationbias