UPDATE: To see how science should absolutely not been done, see our investigation of 4 papers which total over 150 errors.
I am but a phd student, only just starting to crawl my way into academia. Yet even the little ones are allowed to dream big, and this is my dream of how science will be done in the (hopefully near) future.
Everything will be open…
… no matter the publication method. All scientific articles will be freely accessible by anyone, because no matter how many subscriptions your university can or cannot afford, scientists should be able to read everything. Do you know how much money is spend on buying back the right to read the research we wrote? To you give an impression, a small country like the Netherlands (only 13 universities) pays over 42.000.000 euro per year. As science is not just a hobby of scientists: you should be able to access the scientific literature, even if you are not working for a research institute.
With this free access to scientific articles comes free and instant access to the data, analyses, and any other material which underlies the paper. The scientific method is nothing if it isn’t transparent and . How we have come so far while stubbornly keeping relevant information hidden from each other is beyond me. Furthermore, reviewers and editors will become accountable by making the reviews transparent. This will discourage low quality reviews and abuses of power, while encouraging strong reviews. In addition, good reviewers can be properly accredited for their efforts.
Good science flourishes with radical transparency, while bad science withers.
There will be fewer, but larger studies…
… because in science, size does matter. Small, under-powered studies are the hallmark of uninformative science. Currently, everyone seems to want to study their own novel idea, running many small studies. Absurd theories can be kept afloat through a publication system which picks up the few studies which appear informative, but are so by chance alone. Good science is slow science, with large studies which give us so much more informative than had the same amount of resources been split across several smaller studies. Given the limited amount of resources, larger studies means fewer studies. This has the benefit of reducing the workload for reviewers and editors, and lowering both the absolute and relative amount of bad papers. Investing more resources into fewer studies also makes it more feasible to, for example, use multi-level methods and various control groups when necessary.
Every study will be pre-registered…
… or at least to some extent. This will be done at journals, and the study methodology will be reviewed by experts before any data collection takes place. This will prevent countless weak study to ever take place, thereby increasing the overall quality of the published literature. With this single measure, publication bias will almost completely disappear, as publication can no longer be conditional on the outcome.
If there is one way to ruin the scientific literature it is to selectively publish positive findings while ignoring the rest. This is just absurd.
Pre-registration of studies can be done in phases, if necessary, as not every type of study can be fully scripted a priori. These phases, and the decisions made will be published alongside the article, such that the reader can get a good sense of the circumstances in which the data was gathered. Pilot studies can still be done of course, for purposes like testing practical and technical feasibility (but not for informing a power analysis).
There are so many other ways in which science can improve. Making grants smaller and more numerous. Having fewer phd students and more permanent higher level positions. Focusing on replications.
What are your dreams for how science should be done?