The efficacy of anti-TNF in rheumatoid arthritis, a comparison between randomised controlled trials and clinical practice
- W Kievit2,
- J Fransen2,
- A J M Oerlemans1,
- H H Kuper3,
- M A F J van der Laar3,
- D J R A M de Rooij4,
- C M A De Gendt5,
- K H Ronday6,
- T L Jansen7,
- P C M van Oijen8,
- H L M Brus9,
- E M Adang1,
- P L C M van Riel2
- 1Department of Medical Technology Assessment, Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands
- 2Department of Rheumatology, Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands
- 3Department of Rheumatology, Medisch Spectrum Twente, Enschede, The Netherlands
- 4Department of Rheumatology, Sint Maartenskliniek, Nijmegen, The Netherlands
- 5Department of Rheumatology, Rijnstate Hospital, Arnhem, The Netherlands
- 6Department of Rheumatology, Leyenburg Hospital, Den Haag, The Netherlands
- 7Department of Rheumatology, Medical Centre Leeuwarden, Leeuwarden, The Netherlands
- 8Department of Rheumatology, Jeroen Bosch Hospital, Den Bosch, The Netherlands
- 9Department of Rheumatology, TweeSteden Hospital, Tilburg, The Netherlands
- W Kievit, MSc, Department of Rheumatology (470), Radboud University Nijmegen Medical Centre, PO Box 9101, 6500 HB Nijmegen, The Netherlands;
- Accepted 27 March 2007
- Published Online First 10 April 2007
Background: Randomised controlled trials (RCTs) evaluating the efficacy of antagonists to tumour necrosis factor α (TNFα) showed high response percentages in the groups treated with active drugs.
Objective: To compare the efficacy of anti-TNF treatments for rheumatoid arthritis (RA) patients in RCTs and in daily clinical practice, with an emphasis on the efficacy for patients eligible and not eligible for RCTs of anti-TNF treatments.
Methods: First, randomised placebo-controlled trials written in English for etanercept, infliximab and adalimumab for patients with RA were selected by a systematic review. Second, the DREAM (Dutch Rheumatoid Arthritis Monitoring) register with patients starting for the first time on one of the TNF-blocking agents was used. Patient characteristics, doses of medication and co-medication as well as the ACR20 response percentages were compared between RCTs and DREAM data, stratified for trial eligibility.
Results: In 10 of 11 comparisons, the ACR20 response percentages were lower in daily clinical practice than in the RCT active drug group, which was significant in five of 11 comparisons. Only 34–79% of DREAM patients fulfilled the selection criteria for disease activity in the several RCTs examined. DREAM patients eligible for RCTs had higher response percentages than ineligible DREAM patients. ACR20 response percentages of eligible DREAM patients were comparable with the ACR20 response percentages of the RCT active drug group in 10 of 11 comparisons.
Conclusion: The efficacy of TNF-blocking agents in RCTs exceeded the efficacy of these drugs in clinical practice. However, in clinical practice more patients with lower disease activity were treated with TNF-blocking agents compared with those treated in RCTs. For daily practice patients who were eligible for RCTs, responses were more similar to responses reached in RCTs.
Rheumatoid arthritis (RA) is a chronic, progressive inflammatory disease with the potential to cause cartilage destruction and bone erosions.1 To date, the aetiology of RA is unknown. Pro-inflammatory cytokines such as tumour necrosis factor α (TNFα) have been suggested to play a central role in the pathogenesis of the disease.2 Inhibition of TNF has been shown to reduce disease activity and to delay the process of progressive joint damage.3–5 Presently, three different anti-TNF agents are available for patients with RA: etanercept (Enbrel®), infliximab (Remicade®) and adalimumab (Humira®).
Randomised controlled trials (RCTs) on anti-TNF show high response percentages. It is suggested that comparable efficacy is hardly ever achieved in daily clinical practice.6 Differences in efficacy between RCTs and clinical practice might be explained by: patient selection; a wash-out period before inclusion, which artificially increases the disease activity; differences in the doses; co-medication; occurrence of co-morbidity; and adherence.
In the present study, we compared the efficacy of anti-TNF drugs in RA from RCTs with their efficacy in the Dutch Rheumatoid Arthritis Monitoring (DREAM) cohort on anti-TNF (daily clinical practice).
We performed a systematic review of RCTs of anti-TNF agents in RA with an emphasis on efficacy parameters as well as data on dose, co-medication and patients’ characteristics. The RCT data were compared with the data from the DREAM cohort, reflecting daily clinical practice.
Identification of studies
RCTs (phase III studies) were identified from the Medline database (published before the end of 2005) by using the search strategy for RCTs described in Egger et al.7 The search strategy was combined with the following terms to identify relevant studies for our purpose: rheumatoid arthritis and ((etanercept or Enbrel or infliximab or Remicade or adalimumab or Humira) or tumour necrosis factor or TNF)
Based on the title and abstract, all studies that compared etanercept, infliximab or adalimumab with a placebo in the treatment of RA were included, regardless of the concomitant use of methotrexate (MTX). We focused on studies that evaluated treatment groups with a comparable dose and frequency as labelled in the Netherlands (40 mg adalimumab once/2 weeks, 25 mg etanercept twice weekly and 3 mg/kg infliximab per 8 weeks). Only articles written in English were included. Final inclusion and exclusion decisions were made after the articles had been examined. If more than one article from the same study was found, the first article published was included.
The included trials were evaluated with respect to patient characteristics, dosage of anti-TNF, MTX and co-medication, and efficacy parameters using predefined data entry forms. Different total joint counts were reported in the articles and the daily clinical practice database. For comparison of baseline characteristics, reported joint counts were converted from number of joints into percentage of joints. The primary efficacy outcome was the percentage of patients with an ACR20 response in the active drug group and in the placebo group.
Daily clinical practice data
In April 2003, a register was started to monitor and evaluate prospectively the use of anti-TNF in patients with RA in 11 hospitals in the Netherlands, the DREAM study on anti-TNF. In the Netherlands, patients are allowed to start with any anti-TNF therapy if they meet the following criteria: (1) diagnosis of RA (according to ACR criteria, 19878); (2) disease activity score (DAS28) >3.2;9 and (3) previous treatment with at least two other anti-rheumatics including MTX at an optimal dose (maximum dose of 25 mg/day) or intolerance for MTX. All RA patients in the 11 hospitals starting on anti-TNF for the first time were included in the DREAM register. Patients were treated at the discretion of the attending physician.
Independent trained research nurses assessed patients every 3 months and collected data on patients’ demographics, disease activity, treatment, dosages and adverse events. Disease activity was measured using ‘core set’ measures: 28-joint count for tender and swollen joints, erythrocyte sedimentation rate (ESR), C-reactive protein (CRP) level, the Health Assessment Questionnaire (HAQ), and visual analogue scales (VAS) for general health, disease activity and pain. Additionally, information on patient characteristics and therapeutic setting was available in this register.
To compare the patient characteristics, the following variables were analysed: dosage, disease duration, age, gender, rheumatoid factor, percentages of tender and swollen joints, number of prior disease-modifying antirheumatic drugs (DMARDs), and concomitant DMARD, corticosteroid and non-steroidal anti-inflammatory drug (NSAID) use. In order to obtain an indication of the relevance of differences in baseline values between RCTs and DREAM, mean values and SEMs were calculated on the basis of SDs as presented in the articles. Because the joint counts were converted from number of positive joints into percentage of positive joints, SEs of the percentage of joints affected were calculated as the SE of a proportion.
Because the physician global assessment was not present in the DREAM register, modified ACR20 response percentages were calculated as the primary outcome. Modifications were done in two ways. First, the ACR20 response was calculated as a 20% improvement in four out of six parameters, giving a overestimation of the percentage of patients with a response. Second, an underestimation was calculated as a 20% improvement in five out of six parameters. Both ACR20 response percentages are presented in this paper. All efficacy data were analysed as intention-to-treat analyses with a non-responder imputation.
Differences in ACR20 response percentages between the RCTs and the daily clinical practice data were statistically tested for every single RCT. In order to correct for multiple testing of the same hypothesis, we adjusted the significance level by the Bonferroni correction. We hypothesised that the response in daily clinical practice will be less impressive than in the RCTs. Therefore, focus will be on the most conservative comparison of the overestimation of ACR20 response with the RCT active drug response.
The percentage of patients in daily clinical practice eligible for the RCTs on the basis of the RA activity was calculated for each study. Furthermore, groups of eligible and ineligible patients were compared with the RCT active drug group with regard to the overestimation of the percentage of patients with an ACR20 response.
All analyses were performed using SPSS 12®.
The search strategy yielded 492 records. On the basis of title and abstract, a total of 27 potentially relevant papers were selected and retrieved to obtain more detailed information. Of these 27 papers, a further 15 were excluded: 13 phase I or phase II studies and two studies of early RA patients. Of the remaining 12 papers, five concerned etanercept,4 10–13 two concerned infliximab14 15 and five concerned adalimumab.16–20 One study14 18 was excluded from the comparison of efficacy because only the response according to the Paulus criteria was presented in the article. All studies except one (Furst et al18) used a wash-out period of 4 weeks for all DMARDs except MTX in the add-on studies. The follow-up time in the selected studies ranged from 12 to 30 weeks.
By December 2005, 546 patients had been included in the register. Five treatment groups were observed: infliximab with MTX (n = 103), etanercept with MTX (n = 171) and without MTX (n = 45), and adalimumab with MTX (n = 186) and without MTX (n = 31). For the infliximab patients, the mean time of follow-up was 20 months; for all other patients it was 13 months. Baseline characteristics are presented in table 1.
The percentage of patients stopping anti-TNF treatment within 6 months (maximal follow-up time of the included articles) differed for the various treatment approaches: 6.8% stopped in the adalimumab with MTX group, 41.9% in the adalimumab monotherapy group, 11.4% in the etanercept with MTX group, 26.7% in the etanercept monotherapy group and 16.5% in the infliximab with MTX group. However, the reasons for stopping—that is, adverse events and lack of effectiveness, were comparable in all groups.
Only minor differences in patient characteristics and ACR core set baseline values between the RCTs and DREAM study were observed (table 1 and table 2). CRP levels, tender joint counts, HAQ, VAS pain and VAS global values were significantly lower in the DREAM data compared with both Van de Putte trials1920 .
Anti-TNF and MTX dosage as well as the use of NSAIDs were comparable. Between 29% and 54% of DREAM patients used corticosteroids, whereas the corticosteroid use in RCTs ranged from 44% to 69%. In RCTs, the prednisone dose was limited to a stable maximal dose of 10 mg/day. In the DREAM patients using corticosteroids, the baseline prednisone dosage was approximately 10 mg/day, but 40% of these patients stopped using it after starting anti-TNF.
Figure 1 presents a graphical display of the effects of anti-TNF on the ACR20 response in DREAM patients, as well as in the RCT active drug group and in the placebo group. The ACR20 response percentages are generally lower in daily clinical practice than in the RCT active drug group. This difference is significant in five of 11 comparisons with an overestimation and in nine of 11 comparisons with an underestimation (table 3). The absolute difference between the RCT active drug group and daily clinical practice varied between 2% and −44% for overestimation and between −11% and −56% for underestimation of ACR20 response percentages. The difference in responses was smallest for adalimumab and largest for etanercept.
Although our results presented in table 1 indicated that the baseline values and patient characteristics were comparable between the RCT and the DREAM population, table 4 shows that only 34–79% of DREAM patients fulfilled the inclusion criteria for baseline disease activity in the RCTs.
Figure 2 presents a graphical display of the effects of anti-TNF on the overestimation of the ACR20 response in DREAM patients eligible and ineligible for the RCTs as well as in the RCT active drug group and in the placebo group. The number of eligible or ineligible patients is very small in some comparisons (see table 4), giving rise to large standard errors. The ACR20 response percentages in DREAM patients eligible for the RCTs were generally still lower than the response percentages in the RCT active drug groups (fig 2). The difference between the RCTs and eligible DREAM patients was statistically significant in one of 11 comparisons (table 5). The absolute difference between the RCT active drug group and eligible DREAM patients ranged from 14.7% to –35%. The absolute difference between the RCT active drug group and ineligible DREAM patients ranged from –9.4% to −54.8% and was significantly lower in six of 11 comparisons.
DREAM patients who were eligible for the RCTs had higher response rates than ineligible patients. The absolute difference between the eligible and ineligible patients ranged from –9.6% to 44.2% in favour of the eligible patients, but was significantly higher in two comparisons.
Our data confirm the impression that in clinical practice the effects of anti-TNF treatment are smaller than in published RCTs. In five of 11 comparisons there was a significant difference between the daily clinical practice data and the active drug group of the RCTs. Further, the data indicate that selection towards high disease activity in RCTs is a major explanation for the observed difference in efficacy. This can be concluded from the fact that the differences in response between the active drug groups and the eligible patients were smaller than the differences between the active drug groups and the ineligible patients. Furthermore, eligible patients had up to 44% higher response rates than ineligible patients.
With respect to dosing regime and co-medication, a difference in the use of corticosteroids between daily clinical practice and RCTs was observed. In clinical practice, fewer patients used prednisone and many patients stopped prednisone after starting anti-TNF therapy. This might be another explanation for the lower efficacy of anti-TNF in clinical practice compared with the efficacy in RCTs.
Our results confirm the observation by Sokka and Pincus,21 who showed that most patients receiving routine care did not meet the inclusion criteria for the early RA trial of etanercept (ERA) and the ATTRACT study (42% and 5%, respectively, did meet the criteria). However DREAM patients fulfilled the inclusion criteria for disease activity more frequently, which is probably explained by the fact that their routine care cohorts consisted of all RA patients instead of RA patients who started anti-TNF therapy. Zink and colleagues also showed that eligible patients had higher response rates than non-eligible patients.22
Wolfe and Michaud concluded that the design of RCTs exaggerates the anti-TNF treatment effect due to a wash-out, patient selection and regression to the mean.23 This finding is confirmed by our result showing that daily clinical practice patients eligible for the RCTs have a larger response than patients ineligible for the RCTs. Wolfe and Michaud suggested that the efficacy of new drugs observed in RCTs should be corrected for the active comparators by subtracting the placebo response from the response in the RCT active drug group.23 This could be possible if the clinical effect of the placebo itself is zero. Is has been proven that this is not the case in subjective continuous outcomes, especially measures of pain.24 Five out of seven ACR core set measures are subjective outcome measures or consider pain. Therefore, the placebo response is a combination of the placebo effect itself and other effects, such as patients’ preferences and regression to the mean. These placebo effects are different in every trial and observational setting, therefore it is not possible to develop an algorithm to correct the efficacy shown in RCTs for the expected effectiveness in daily clinical practice. Therefore, we illustrate the difference between clinical practice and RCTs by describing the possible confounding issues and their magnitude as observed.
This study has limitations. For our data collection we only counted 28 joints instead of 68 as in most RCTs. This might result in an overestimation of the baseline disease activity in the observational data because the 28 counted joints are likely to be the 28 joints most affected in RA.25 Next, patients in daily clinical practice are treated with the medication of preference. We consider it probable that this can result in a larger treatment effect than in RCTs.26 27 We were unable to calculate the exact ACR20 response criteria as was done in most selected papers. Instead, we had to compare the efficacy of anti-TNF on an overestimation or underestimation of the ACR20 response, which makes interpretation more difficult.
RCTs are the appropriate design to evaluate efficacy of new interventions. However, observational phase IV studies have a complementary value to investigate long-term side effects and efficacy, and may be useful to study effects in patients not typically included in phase III RCTs.26
This study confirms the impression that in clinical practice the effects of anti-TNF are smaller than in published RCTs. For daily practice patients who were eligible for RCTs, responses were more similar to responses reached in RCTs. Responses were lower in patients ineligible for RCTs. Selection towards high disease activity and the continued use of co-medication in RCTs are probable explanations for the difference in effects of anti-TNF in clinical practice and in RCTs.
We are indebted to all research nurses and rheumatologist of the 11 departments of rheumatology for their participation and contribution in the data collection: J Alberts, P Barrera Rico, M Creemers, J Deenen, A van Ede, T van Gaalen, E de Groot, H van Heereveld, F van den Hoogen, R Laan, P van Riel, L Schalkwijk, C Versteegden, C Vogel, M Vonk (Radboud University Nijmegen Medical Centre), H Cats, A Eijsbouts, M Franssen, I Geerdink, S Hol, F van den Hoogen, M Jeurissen, P Koelmans, P van ’t Pad Bosch, D de Rooij, A Stenger, H van Wijk, (Sint Maartens Kliniek, Nijmegen), T Berends, C Bijkerk, C De Gendt, J Harbers, M Janssen, A de Jong, H Knaapen, H Visser (Rijnstate Hospital Arnhem), A ter Avest, K Drossaers, M Hoekstra, M Kruijssen, I Kuper, M van de Laar, A Mooij, H Vonkeman (Medisch Spectrum Twente, Enschede), H Bernelot Moens, E Bos, K Drossaert, C Haagsma, K van de Hoeven, J Oostveen (Twenteborg Ziekenhuis Almelo), M Kleine Schaar (Streekziekenhuis Midden Twente, Hengelo), J de Boer, H van de Brink, S Erasmus, J Moolenburgh, W Swen (Medical Centre Alkmaar, Alkmaar), I Henkes, H Hulsman, K Ronday (Leyenburg Hospital, Den Haag), M Geurts, J Haverman, P van Ooijen, N Wouters (Jeroen Bosch Hospital, Den Bosch), GAW Bruyn, EN Griep, PM Houtman, TL Jansen, A Spoorenberg, A Krol, J Woudwijk (Medical Centre Leeuwarden, Leeuwarden), R van Berkel, H Brus, W Hissink Muller, A van Roy, M. Wijnands (Twee Steden Hospital, Tilburg).
Competing interests: None.
Funding from the Dutch National Health Insurance Board and the Dutch affiliations of Wyeth Pharmaceuticals, Abbott Pharmaceuticals and Roche Pharmaceuticals enabled the data collection for the DREAM study.
- C-reactive protein
- disease-modifying antirheumatic drug
- Dutch Rheumatoid Arthritis Monitoring
- erythrocyte sedimentation rate
- Health Assessment Questionnaire
- non-steroidal anti-inflammatory drug
- rheumatoid arthritis
- radomised controlled trial
- tumour necrosis virus
- visual analogue scale