Arguing for all the wrong reforms

You have been p hacked

Calls to reform the way we do science are becoming increasingly more frequent. Most scientists seem to agree that we should fight problems such as p hacking, publication bias, and corrupting incentives. The key question is how we should do this, but what makes a reform effective? On September 1, a quantitatively impressive group of people … Read more

How (not) to interpret confidence intervals

What are confidence intervals? It is the time interval between when you have successfully learned what confidence intervals mean and when you start to realize that you’ve never fully understood them. As a master student I had to read Geoff Cumming’s “New Statistics” book, which prominently featured confidence intervals. To this day I still don’t … Read more

What are long-term error rates and how do you control them?

I stumbled upon an article which used a Bonferroni correction to ‘control’ family-wise error rates. While this isn’t shocking by itself, I was happily surprised by how they applied it. This is what they wrote: “In order to account for multiple comparisons, statistical significance was set at a p-value of less than 0.0056 (.05/9 tests) using … Read more

A real life case study on reporting inconsistencies: what would you do?

(Note: If you haven’t done so yet, be sure to read my earlier blog post as an introduction to the 4 papers with 150+ inconsistencies) Many scientists will at some point in their academic career play a game about research ethics involving discussions of case descriptions. These cases typically start with a description of a tricky scenario, for … Read more