Fig 1. Relative risk of showing benefit or harm of treatment by year of publication for large NHLBI trials on pharmaceutical and dietary supplement interventions.
Positive trials are indicated by the plus signs while trials showing harm are indicated by a diagonal line within a circle. Prior to 2000 when trials were not registered in clinical trials.gov, there was substantial variability in outcome. Following the imposition of the requirement that trials preregister in clinical trials.gov the relative risk on primary outcomes showed considerably less variability around 1.0.
We have posted before about the rise of preregistration for experimental studies. Justin has started doing this for his studies and is seeing a few clear benefits. First, it disciplines your study so that you plan it out better. Having to preregister is an up-front pain, but makes the study better. Second, there is a rapid increase in referees and journals looking for preregistration. If you have an interesting result, especially relying on interaction or heterogenous effects, you really need a pre-registration to be credible. Finally, good science should have us sometimes finding null results. If you pre-register, it is a little harder for refers to argue the study should not be published because it doesn't find anything -- you force them to consider the design and not just the results.
