The Importance of Publishing Null Results: An Effort Towards Open Science
As we have previously mentioned, Null Results Publishing saves researchers time, money, and effort (In the post below). By publishing these results, we can avoid wastefulness of repeating experiments that are already well-established. But why are so many valuable null results not being published?(Read below post)
In the past few days, our IBSAglobal team has conducted interviews with researchers and experts to find out about the challenges to the failure to publish null results. Based on these interviews, we were interested in knowing why researchers hesitate or struggle to publish their null results. In this blog post, we shall summarize the facts we gathered from these interviews and the challenges we wish to highlight.
Post>>
IBSAglobal’s Intention: Making Null Results Publishing Easier and Favorable
At IBSAglobal, we hope to create a new publishing paradigm that works for researchers to easily report their null results. We think that science cannot be trendy or based on positive results only. It is about finding the truth—be it the truth that supports the hypothesis or otherwise. Through our website, we hope to offer researchers an opportunity to record these precious truths and let science move forward with absolute transparency, not what is “trendy.”
International Bureau for Scientific Advancement (IBSAglobal): Student-Led Initiative for Promoting STEM
Our staff in IBSAglobal is committed to overcoming STEM obstacles and finding new solutions. We are convinced that scientific progress relies on collaboration, transparency, and readiness to meet and exchange all results, positive or negative.
Professional's and Researcher's insights & stories
#1: Null results DOES get cited
One of the researchers shared their experience of having their Master’s work null results paper published. They had to wait for an incredible three years and five journal attempts before the paper was accepted. Despite the experience, the paper ended up being one of their third most cited papers.
The key takeaways from this story are:
Publishing Null Results is Not Easy: Null results papers get rejected by journals, and thus it becomes more difficult for researchers.
Long-Term Impact: Null results are not necessarily desirable in the short term but, in the long term, can be strong and beneficial to the scientific world.
#2: Issues in field where null results are acknowledged
In physics as well, null results are reported, though they are not problem-free. One researcher outlined how, while null results are acknowledged, the problem is that such results do not provide a simple route for follow-up research. Positive results, on the other hand, advance the field and are rewarded.
Major takeaways from this experience are:
Null Results Are part of the process: While null results are essential, they are not short-term progress, and therefore positive results are more appreciated in scientific progress.
Smart Planning and Framing: With smart planning and framing of research questions, null findings can be framed as worthwhile contributions to the field.
Academic Evaluation is Flawed but Effective: The current system, driven by measurement and evaluation, is certainly not perfect, but it generally works, even as it proves challenging to introduce better options.
#3. Clinical Trials, Incentives, and the Requirement for Publishing Reform
In clinical trials, some progress has been made towards publishing null results, especially in large trials. But one scientist complained about the present academic system, where academics are rewarded for quantity—writing lots of papers—instead of making big, impactful discoveries.
Key observations are:
Need for Flexible Formats of Publishing: The author proposed an ideal scenario where publishing would be more flexible with different formats such as lab reports, rapid publications, commentaries, and media based on existing technology. It would accommodate a broader variety of results, including null results.
Hope for Reform: In spite of the difficulties, the researcher is optimistic that existing disruptions to the academic system will spur meaningful reforms that promote more honest, diverse forms of publishing research, including null results.
#4. Publishing Null Results and the Failings of the System
Another scientist said that there are certain papers that just necessarily have to end on “nothing interesting”—and that’s fine. Nature Communications and Scientific Reports, for example, do accept null results, but the larger issue is the culture of academia that pays out in citations and hype rather than the value of research findings.
Key findings are:
Publishing Null Results Is Possible, But Penalized: Although a few journals do publish null results, the system as a whole penalizes researchers for publishing them. Null results are “low value” since they don’t generate the sort of attention positive or novel results do.
Career Development Hinges on High-Impact Papers: Career advancement in most researchers significantly relies on highly cited, high-impact papers. Therefore, the incentive is to produce something that will generate hype and not follow dull null results.
Reform Is Not Imminent: The writer is skeptical that substantive reforms will happen at some point in the near future, as the incentives for reporting negative results are heavily committed to career preservation and the predominant academic system’s emphasis on visibility and citations.
#5. Journal Bias and Funding System Challenges
The primary reason for the difficulty in publishing null findings is the journal’s and the funding agencies’ preferences. Journals, because they want attention and citations, will naturally favor positive findings that result in new discoveries. This is because positive findings have the tendency to draw enormous attention.
In the same way, universities and funding agencies will similarly fund “promising” studies. Null results are not promising, so they don’t get similar funding or attention, and are thereby ignored.
In Plain Language:
Journals and funders want to hear buzz, rather than fact. And so they prefer sexier, positive outcomes to dull null results.
#6. Practical Problems with Publishing Null Results
One of the professors explained the difficulty they themselves faced in trying to publish a student’s null result. Although the research itself was excellent, they faced a number of problems. Firstly, it is not easy to persuade a journal to publish the null results. Secondly, the researcher would have to produce rigorous evidence that the results were authentic and not the product of faulty methodology, and this took a lot of time and expense.
Even with all such attempts, the impact of the paper would also be low. Scientific Reports and PLoS One are a few of the journals that welcome scientifically valid work but levy extremely high article processing charges (APCs) for open access.
In Simple Terms:
Even if it’s a null finding and scientifically sound, it’s a difficult paper. It’s a lot of work to show the finding, and even then it doesn’t get read much. There are a few journals that do publish these articles, but they pay a lot. And also, scientists may be motivated to work on exciting new discoveries rather than completing “boring” negative studies.
Additional point:
Another argument presented is the inclination of scientists to gravitate towards flashy new research, rather than investigating “boring” null result research. A solution proposed is pre-registration of studies where researchers pre-register the hypothesis and method before they even run the experiment. The journal then commits to publishing the result, whether positive or negative, and it is a more open and equitable system.
Pre-registration is currently being used on a daily basis in clinical trials but not yet widely used in basic science.
What's next?
Soon IBSAglobal intent to come with innovative journal model and structure to ensure nothing is left out in path of advancement.
