Why results of clinical trials and observational studies of antitumour necrosis factor (anti-TNF) therapy differ: methodological and interpretive issues

Abstract

Objective: Results of randomised clinical trials (RCTs) appear to differ from results of observational studies. This paper explores differences in methodology, interpretation and presentation of results that elucidate these differences.

Method: We identified patients who completed a survey questionnaire during the period January 1998 through December 1998 and also completed one between July 2003 and June 2004, an average span of 4.7 years. The mean time from study initiation to anti-TNF administration was 2.1 years, and the mean treatment time was 2.1 (SD 1.3) years at study closure. During this period 38.3% of patients received anti-TNF therapy. We compared the results of patients in this group with results from RCTs.

Results: RCTs utilise flare design, patient selection, control groups and regression to the mean. Observational studies, on the other hand, confound additional prior therapy and anti-TNF effect, do not employ control groups, and may have less regression to the mean.

Conclusions: RCTs and observational studies assess and report efficacy and effectiveness in ways that are so different that they are often incommensurable. A key difference is whether results should represent changes from flare states or should, instead, consider chronic status prior to initiation of therapy. There is little evidence that the clinical state at the start of most anti-TNF RCTs represents a chronic state. Economic analyses that utilise the RCT starting point overestimate the cost effectiveness of anti-TNF therapy. The solution for these problems and a guide to understanding the real results of anti-TNF therapy lies in collecting preclinical trial data in all patients who will enter clinical trials. In addition, RCT results would more approximate those of observational studies if all reporting was done after subtracting the effect of the comparator group.