by Caitlyn A. Hall, Andrea Popp, Hannes Müller, and Tim van Emmerik
Understanding and learning from unexpected results is a fundamental element of science. Different names exist for these results, e.g., failures, obstacles, or unexpected results. Although all of these names sound unexpected, they are important for the understanding of processes, developing and testing of theories, and identifying pitfalls and possible dead-ends in science.
By carefully designing and conducting experiments with some level of trial-and-error, researchers eventually find results that will be published in peer-reviewed scientific journals. Paradoxically, we typically only publish the successful tests and their results. What comes of the weeks to months of critical information that led to this successful experiment? It usually remains in the dark. However, not sharing unsuccessful iterations or unexpected results — defined here as experiments that do not adequately confirm an accepted hypothesis, despite sound and careful experimental design, planning, and execution — along the way prevents others to learn from these endeavors (Nature Editorial, 2017).
In the past, many philosophers, including Popper (1963) and Chalmers (1973), have emphasized that science can only advance by learning from mistakes. Moreover, recent literature in various fields elaborate on the many benefits and values of publishing unexpected results and call upon the scientific community to nurture their dissemination (e.g., Andréassian, et al., 2010; Schooler, 2011; Matosin et al., 2014; Granqvist, 2015; Goodchild van Hinten, 2015; Boorman, et al., 2015; PLOS collections; 2015, 2017; Nature Editorial, 2017). Despite the various calls to report such results and the frequency they occur in the labspace, they are still underrepresented in most fields of our current publication system. The reasons can be manifold such as, a lack of incentive (no scientific reward) or the fear of a negative reputational impact.
So, why should you report your failed approaches and unexpected findings?
By reporting on unexpected findings, we can do the following:
- Decrease the currently existing publication bias towards positive results
- Save time and resources of other scientists exploring same/similar hypotheses and/or approaches
- Increase transparency and reproducibility of our studies
- Share all findings of publicly funded projects
How and where can you share your unexpected findings?
You can share your unexpected results at:
- Special journal issues
- Dedicated sessions at conferences
- Platforms (e.g., Researchgate)
- Supplementary material of your paper
- Blog posts
We aim to stimulate this discussion via the new Young Hydrologic Society collection “Unexpected Results in Hydrology”. We want to instill a positive perception to change the way in which the scientific hydrologic community value unexpected and negative results including individual researchers, scientific societies, funding agencies, and publishers. Therefore, we invite researchers to report their negative and unexpected results, such that we are able to holistically advance science – by sharing our failures, not only our successes.
Reporting on such findings should include the following components in a maximum of 3,000 words: 1) an original research objective and expected results, 2) a brief summary of experimental design and methods, 3) discussion on the experimental results and challenges, including images and/or figures, and possibly 4) lessons learned and the path forward.
After a peer-review done by the editors of this collection, the post will get a DOI and will be visible on the YHS website and on a dedicated ResearchGate project site. On ResearchGate we invite discussions on published submissions such that the authors can receive feedback to facilitate new insights from the scientific community. Upon enhancing their previous analysis or coming to new conclusions, we welcome resubmissions by the original authors.
References
Andréassian, V., Perrin, C., Parent, E. and Bárdossy, A. (2010). Editorial – The Court of Miracles of Hydrology: can failure stories contribute to hydrological science? Hydrol. Sci. J. 55(6), 849–856.
Boorman, G.A., Foster, J.R., Laast, V.A. and Francke, S. (2015). Regulatory Forum Opinion Piece: The Value of Publishing Negative Scientific Study Data, Toxicol Pathol, 43(7), 901-906. doi: 10.1177/0192623315595884
Chalmers, A.F. (1973). On Learning from Our Mistakes, The British Journal for the Philosophy of Science (Oxford University Press) 24(2), 164-173.
Goodchild van Hilten, L. (2015).Why it’s time to publish research “failures”-Publishing bias favors positive results; now there’s a movement to change that. Elsevier Connect.
Granqvist, E. (2015). Why science needs to publish negative results. Elsevier Connect
Matosin, N., E. Frank , M. Engel, J.S. Lum, and Newell, K.A. (2014). Negativity towards negative results: a discussion of the disconnect between scientific worth and scientific culture. Disease Models and Mechanisms. 7(2): 171–173. doi: 10.1242/dmm.015123
Nature Editorial (2017). Nurture negatives, Nature 551, 414, doi: 10.1038/d41586-017-07325-2
PLOS Collections (2015). Positively Negative: A New PLOS ONE Collection focusing on Negative, Null and Inconclusive Results, PLOS ONE Community Blog
PLOS Collections (2017). Negative Results: A Crucial Piece of the Scientific Puzzle, PLOS ONE Community Blog
Popper, K. (1963). Conjectures and Refutations: The Growth of Scientific Knowledge, Routledge Classics, London and New York
Schooler, J. (2011). Unpublished results hide the decline effect, Nature 470, 437, doi:10.1038/470437a
