Increasing concern has been expressed that biomedical research findings many times fail to be replicated and validated. This may occur across the spectrum from basic research to clinical applications. Small studies, poor design, the play of chance, exaggeration in early claims, data dredging, selective publication and reporting of information, strong expert opinions, biases, and financial or other conflicts of interest all probably combine to create uncertainty about whether research findings will stand the test of time and independent replication. Some research findings or beliefs are not even questioned and no effort is ever made to challenge them, but this does not mean that they are true. The recent advent of molecular medicine has only increased the complexity and number of questions that may be asked. It is unclear though, whether the information derived from the various fascinating discovery-driven approaches, massive as it is, is more likely to be possible to replicate. The ratio of sample size to scientific question asked has been decreased to infinitesimal levels, and uncertainty remains as high as ever.

SMALL SAMPLE SIZES: EMPIRICAL DATA

For rheumatology and autoimmune diseases, the danger is as great as for any other medical discipline. Much of the replication difficulties may have to deal with the fact that claims are made or refuted based on limited data in single studies. The median number of patients in randomised controlled trials in systemic sclerosis is only 28,¹ exactly the same as for trials on treatments for systemic lupus erythematosus.² Including also trials on common diseases like rheumatoid arthritis or osteoarthritis, the median sample size for a rheumatology related trial has barely improved, moving from 46 to 88 patients in the past decade.³ Studies in the sample size ranges encountered in cardiology or oncology mega-trials …