Assessing the Transparency of Methods in Scientific Reporting

Transparency in reporting methods consists of clear and comprehensive documentation of the processes, techniques, and procedures employed during the study. It involves providing detailed explanations of the population and sampling procedures, the manipulations and measurements used, and other relevant aspects of the research process. A simple approach to assess the transparency of the reporting of the methodological aspects of a study could be, for example, to ascertain whether another researcher, upon reading the article and utilizing the materials provided by the author (e.g., supplementary materials online, links to repositories containing stimuli, etc.), would have all the necessary information to replicate the study from scratch, in a manner very similar to how the original study was conducted. If a scientist, based on the paper’s reading and available materials, doesn’t have sufficient information to conduct a replication, this means that the reporting doesn’t describe the methods fully transparently. One could argue that when a paper lacks methodological details, readers can reach out to the authors and request additional information. However, this approach is far from optimal. Firstly, it is not always successful: As it will emerge from the present research, it is not consistently clear to whom one should address these inquiries; moreover, at times the corresponding author is no longer active in academia; finally, we are not aware of studies that specifically addressed sharing research materials upon request, but personal experience show that requests for sharing materials may go unanswered or receive a negative response (e.g., due to technical issues like a broken computer or an office relocation) and studies on data sharing clearly indicated that the request for research data from authors in a substantial proportion of cases proves unsuccessful (see e.g., Stodden et al., 2018). Secondly, even when authors do answer the request and share the materials, it usually entails a time-consuming process. This work was stimulated by our informal observation, which we shared with colleagues: When attempting to replicate a scientific experiment, the scientific report of the original study often lacks crucial information. Moreover, transparency in the methods is vital not only for replication. It is also a key preliminary requirement for understanding and evaluating research, expanding its results, and comparing studies. A transparent methods section might contribute to bolstering the overall credibility of the research.