Registered Reports

What are Registered Reports?

Registered Reports are scientific articles which are peer-reviewed prior to any data collection. In practice this means that authors write the Introduction and Method section of a paper and send this to a journal, together with all relevant materials and analyses scripts. Peers will then review the manuscript, highlight any changes that need to be made, and decide to decline or provisionally accept the manuscript for publication. After the authors have finished data collection, data analyses, and have written the Results and Discussions sections they again send it the manuscript in for review. As the manuscript was already provisionally accepted, it will be published based only on the condition that the authors have followed their protocols.

For more information, see also this overview made by the Center for Open Science

What are the strengths of Registered Reports?

The manner in which data was collected and analyzed heavily influences it’s (evidential) value. When decisions regarding analyses and publication are conditional on the data (e.g., reporting or deciding to publish a result because it fits with a theory) this distorts the scientific literature. A powerful way to circumvent this source of bias is making such decisions before the results are known.

This format is designed to reward best practices in adhering to the hypothetico-deductive model of the scientific method. It eliminates a variety of questionable research practices, including low statistical power, selective reporting of results, and publication bias, while allowing complete flexibility to report serendipitous findings.

What is the difference between Registered Reports and pre-registration?

When someone talks about pre-registration, they are typically talking about independently pre-registering the methodology of a study at a website such as AsPredicted.org or the Open Science Framework. Of course, this pre-registration should occur before any data is collected. You can show the time-stamped pre-registration when you submit your study to a journal. Pre-registration is a good way to severely limit the (potential) influence of Questionable Research Practices (QRPs) such as p-hacking and outcome switching.

Registered Reports go much further. Not only are the methodology effectively pre-registered, they are also peer-reviewed and (provisionally) accepted for publication, prior to any data collection. As such, they are not only resistant against QRPs such as p-hacking and outcome switching, but they also effectively eliminate publication bias. This means that, as a reader, you can be very confident in the results of a Registered Report.

Note that the Registered Report format is not limited to replication studies but is also highly suitable for novel research.

Which journals accept Registered Reports?

The Center for Open Science maintains a list of all journals accepting Registered Reports.

They also have a Zotero Library of all published Registered Reports; you can also find all pre-registered studies.

I am part of a task force which contacts journals to convince them to start accepting the Registered Report format as a publication method. Let us know if you want to help!

I also have a Twitter bot @RegReports which regularly tweets about Registered Reports. Please follow and re-tweet!