Efficacy of biological agents in juvenile idiopathic arthritis: a systematic review using indirect comparisons
- 1Department of Pediatrics/Pediatric Rheumatology, Erasmus MC Sophia Children's Hospital, Rotterdam, The Netherlands
- 2Department of Epidemiology and Radiology, Erasmus MC, Rotterdam, The Netherlands
- Correspondence to Dr Janneke Anink, Department of Pediatrics/Pediatric Rheumatology, Sp 1547, Erasmus MC Sophia Children's Hospital, PO Box 2060, 3000 CB Rotterdam, The Netherlands;
- Accepted 20 October 2012
- Published Online First 21 November 2012
Objective Over the past decade, the availability of biological agents for the treatment of juvenile idiopathic arthritis (JIA) has increased substantially. Because direct head-to-head trials comparing these agents are lacking, we indirectly compared their efficacy.
Methods In a systematic review, all available efficacy data from randomised controlled trials performed in JIA with inclusion of biological agents were retrieved. Indirect between-drug comparisons (based on Bucher's method) were conducted only if trials were comparable with regard to design and patients’ characteristics related to treatment outcome.
Results We identified 11 randomised controlled trials. On the basis of the equality of the trials, six trials were grouped into two networks of evidence. Network 1 included withdrawal trials which evaluated etanercept, adalimumab and abatacept in polyarticular course JIA. Indirect comparisons identified no significant differences in short-term efficacy. Network 2 indirectly compared trials with a parallel study design investigating anakinra, tocilizumab and canakinumab in systemic JIA; no differences in comparative efficacy were identified. Although the two networks were constructed on the basis of comparability, small differences in trial design and case mix still existed.
Conclusions Because of the small number of trials and the observed differences between trials, no definite conclusions could be drawn about the comparative effectiveness of the indirectly compared biological agents. Therefore, for now, the paediatric rheumatologist has to rely on observational data and safety, practical and financial arguments. Comparability of future trials needs to be improved, and head-to-head trials are required to decide on the best biological treatment for JIA.
Since 1999, the treatment of juvenile idiopathic arthritis (JIA) has been extended with a new category of drugs: biological agents that target different cytokines and different steps in the immune response. Etanercept, a tumour necrosis factor (TNF)-α receptor antagonist, was the first biological agent approved for the treatment of polyarticular JIA. At present, infliximab (TNF-α antibody), adalimumab (TNF-α antibody), anakinra (interleukin (IL)-1 receptor antagonist), canakinumab (IL-1 antibody), rilonacept (IL-1 receptor antagonist), tocilizumab (IL-6 receptor antibody) and abatacept (selective T-cell costimulation modulator) are also available options or under investigation for the treatment of JIA. Physicians involved in the treatment of JIA have an increasing number of biological treatment options and choosing between them is often difficult.
The efficacy of each agent has been described in one or more randomised controlled trials (RCTs), but head-to-head trials comparing agents directly are still lacking. A few studies have compared the short-term efficacy of biological agents in patients with rheumatoid arthritis (RA) using indirect comparison methods.1–6 This technique allows comparison of two biological agents indirectly (ie, a trial comparing treatment A vs comparator C and treatment B vs comparator C results in a comparison of A vs B), while preserving the randomisation of the originally assigned patient groups.2 ,7 Efforts to indirectly compare these agents in JIA are to our knowledge still lacking.
We therefore conducted a systematic review and described the RCTs with regard to their design and patient characteristics and, where possible, compared the efficacy of different biological agents indirectly. We hope that these results will eventually guide physicians in their biological treatment choices for JIA.
Patients and methods
Search strategy and selection criteria
A systematic search on PubMed, Embase and Cochrane clinical trials was performed using the terms: (‘JIA’ OR ‘juvenile RA (JRA)’) AND ‘randomised controlled trial’ AND (‘tumour necrosis factor’ OR ‘interleukin-1’ OR ‘interleukin-6’ OR ‘etanercept’ OR ‘adalimumab’ OR ‘infliximab’ OR ‘abatacept’ OR ‘anakinra’ OR ‘tocilizumab’ OR ‘canakinumab’ OR ‘certolizumab’ OR ‘golimumab’ OR ‘rituximab’ OR ‘rilonacept’). See online supplementary text for the detailed search strategy. The search included studies up to and including January 2012. To identify unpublished trials, the trial register clinicaltrials.gov, and abstracts from international rheumatology congresses were searched.
We aimed to include the following studies: RCTs with data on efficacy, comparing a biological agent with control treatment (placebo, synthetic disease-modifying antirheumatic drugs (sDMARDs), or a second biological agent), and including patients with JIA (or the previously used criteria for JRA; any onset category).
Two authors (MHO and JA) independently selected the studies from the search and extracted information from published articles and congress abstracts on design, inclusion and exclusion criteria, medication regimens, baseline characteristics and efficacy results during the double-blind phase. Corresponding authors and involved pharmaceutical industries were contacted for any missing data in the publications. Slides of an oral presentation given at an international meeting provided additional information on a selected publication. The trial quality was assessed independently using the Jadad criteria, a widely used five-point score that appraises the quality of trial reporting. The score assesses trial quality on three aspects: randomisation, blinding and handling of withdrawals and drop-outs.8 If scoring was not unanimous, scoring was discussed with a third person (LWAvS-S). Figure 1 shows the flow of the 685 retrieved citations.
For this systematic review with indirect comparisons, all RCTs conducted with biological agents were included, irrespective of trial design. In the field of paediatric rheumatology, in general, two different types of clinical trial are conducted. The randomised placebo-controlled withdrawal trial design (called ‘withdrawal trial’) consists of an open-label lead-in phase where all patients receive the drug. After this lead-in period, only those patients who respond to treatment enter the double-blind phase and are randomised to remain on the drug or receive placebo. For this withdrawal design, the primary outcome chosen is disease flare. The other type of trial used is a ‘classic’ randomised controlled parallel design. These trials often include concomitant treatment with sDMARDs (eg, biological agent plus methotrexate vs methotrexate only), short duration of double-blind phase, or a rescue regimen to limit the time without treatment.
We identified three major outcomes beforehand: percentage of patients with disease flare and percentage of patients achieving an American College of Rheumatology paediatric (ACRpedi) 30 response or inactive disease. These outcomes are based on changes in the following six variables: physician's global assessment of disease activity on a visual analogue scale (VAS; range 0–100 mm, 0 best score), Childhood Health Assessment Questionnaire (CHAQ; range 0–3, 0 best score) by patients/parents, global assessment of well-being by patients/parents (VAS, range 0–100 mm, 0 best score), number of active joints with arthritis, number of joints with limited motion, and a marker of inflammation (erythrocyte sedimentation rate (ESR) and/or C-reactive protein (CRP)). A disease flare is defined as worsening of 30% or more in at least three of these six variables with an improvement of 30% or more in no more than one variable. An ACRpedi30 response is achieved if three or more of these variables improve by at least 30% from baseline, with no more than one variable worsening by more than 30%.9 For the systemic JIA category, a modified ACRpedi30 response is often used with the addition of absence of systemic features. Inactive disease is defined as a disease state with no active arthritis, no systemic features, no uveitis, normal ESR and physician's global assessment indicating no disease activity.10
Relative risk (RR) and corresponding 95% CI were calculated for each trial independently. An appropriate statistical method for conducting adjusted indirect treatment comparisons is the Bucher method.7 Because in the present study each comparison consisted of two trials (one trial vs the other), random-effects meta-analysis and meta-regression were not applicable. In the Bucher method, the relative efficacy of two treatments (A and B) versus a common control group (C) are compared according to the following formula: LnRR'AB=LnRRAC−LnRRBC (where LnRR=natural logarithm of the risk ratio).
The underlying similarity assumption of the Bucher method dictates that trials may differ on study and patient characteristics not related to the treatment outcome, but, if these characteristics are modifiers of the relative treatment effects, then the estimates of the indirect comparisons are biased. To ensure that the same relative effects of a certain drug could have been expected across included trials in the indirect comparisons, indirect comparisons were conducted only between clinical trials with the same design (withdrawal trial with an open-label lead-in phase vs parallel RCTs), and with inclusion of approximately the same patient group with regard to disease duration at baseline and JIA categories included. The results of the adjusted indirect comparisons are given as RR with 95% CIs. Two-sided p values of < 0.05 were considered significant. Analyses were performed with Stata V.12. For data structure and statistical code in Stata, see online supplementary text.
We report outcomes according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and the ISPOR Task Force on Indirect Treatment Comparisons Good Research Practices.11 ,12 See online supplementary table S1 for the PRISMA Checklist.
A total of 17 citations (including citations of congress abstracts and slides of oral presentation) describing 11 different trials were included. Design and patients’ characteristics of the included trials are described in table 1 (for a detailed description, see online supplementary tables S2 and S3). Four withdrawal trials and seven ‘classic’ RCTs with a parallel study design were included. Overall, eight of the 11 trials met the primary end point in favour of the biological agent. The primary outcome was not met by the infliximab trial, TREAT trial and rilonacept trial.13–15
On the basis of the design and quality of the trials and the characteristics of the patients included, two networks for indirect comparisons were selected, and these are presented in figure 2. Three of the four withdrawal trials (the trials evaluating etanercept, adalimumab and abatacept) included patients with resistant polyarticular course JIA and were compared indirectly (network 1). Polyarticular course JIA (or JRA) was defined in all trials as JIA with five or more active joints at any time during the disease course, and also included patients with systemic arthritis with a polyarticular course. For the details of the categories included, see online supplementary table S2. The fourth withdrawal trial evaluated tocilizumab in only patients with systemic JIA, and 50% of these patients did not have a polyarticular course. In addition, the outcome measure of maintaining a modified ACRpedi30 was different from the other three withdrawal trials. This trial could therefore not be included in the indirect comparison analysis.
Of the seven randomised controlled parallel trials, four only included patients with systemic JIA. These trials compared anakinra, tocilizumab, canakinumab and rilonacept with placebo. No detailed efficacy data of the double-blind phase in the rilonacept trial was given, and rilonacept could therefore not be included in this indirect comparison analysis (network 2). The RCT evaluating infliximab by Ruperto et al13 included patients with polyarticular course JIA and could not be included in either network 1 (withdrawal trials) or network 2 (systemic JIA). Finally, two RCTs included patients with early JIA (after mean disease duration of 0.16–0.42 year). However, because of significant differences in the inclusion of rheumatoid factor positive patients (36% vs 2%), no valid indirect comparisons could be made.14 ,16
Network 1: withdrawal trials in polyarticular course JIA
For the three comparable withdrawal trials, the disease duration at baseline varied between 3.8 and 5.8 years, and the mean baseline age was between 10.6 and 12.3 years. The percentage of patients with systemic JIA with polyarthritis in the adalimumab withdrawal trial was unclear. Inclusion of systemic JIA varied between the etanercept and abatacept trials (33% and 19% patients with systemic JIA, respectively), but inclusion of rheumatoid factor positive patients was similar across all three trials (21–24%). Seventeen per cent of patients who were previously non-responsive to TNF-α antagonists were included in the abatacept trial compared with none in the etanercept and adalimumab trials. Of the patients who started treatment in the open-label lead-in phase, 64–78% entered the double-blind phase. Baseline disease characteristics of the patients included in the double-blind phase with regard to number of joints with arthritis, physician's assessment of disease activity, and CHAQ scores were comparable between the three trials. Indirect comparisons indicated no differences in efficacy between these three drugs (table 2).
Network 2: randomised placebo-controlled parallel trials in systemic JIA
The three selected randomised placebo-controlled parallel trials compared anakinra, tocilizumab and canakinumab with placebo. The duration of the double-blind phase was 12 weeks for the tocilizumab trial, 1 month for the anakinra trial, and 15 days for the canakinumab trial. The inclusion criteria with regard to disease duration and non-response to oral corticosteroids were similar. Indirect comparisons identified no significant differences between the drugs with regard to the achievement of a modified ACRpedi30 response (table 2).
In this systematic review, we found that, in JIA, 11 trials compared biological agents with placebo or sDMARDs, and no trials compared biological agents directly. Because these trials differed with regard to design and patient characteristics, not all trials could be included in the indirect comparisons. Two networks of similar trials were identified in order to make valid indirect comparisons. For polyarticular course JIA, etanercept, adalimumab and abatacept seem equally efficacious in preventing disease flare after response to treatment. Canakinumab, tocilizumab and anakinra seem to produce comparable improvement in systemic JIA.
To the best of our knowledge, this is the first attempt to indirectly compare the efficacy of biological agents in JIA. The results of indirect comparisons in RA are not conclusive, but generally seem to indicate that TNF inhibitors are more effective than anakinra, and etanercept is more effective than other TNF inhibitors.4 ,30 ,31 Tocilizumab seems to be more effective in RA than TNF inhibitors and abatacept.3 ,32 In the present study, anakinra and tocilizumab were only investigated in systemic JIA.
Indirect comparisons of etanercept, adalimumab and abatacept in network 1 did not identify significant differences in efficacy. However, no definite conclusions could be drawn because of the differences in case mix and design of the trials. In addition, small differences were not likely to be detected because of the small number of patients included in the trials.
Some of the characteristics of the patients in the etanercept trial may be associated with poorer outcome. The mean disease duration before the start of biological treatment was longer (5.8 years in the etanercept trial vs 3.8 and 3.9 years in the adalimumab and abatacept trials, respectively) and the proportion of patients with systemic JIA was higher (33% in the etanercept trial vs 19% in the abatacept trial). In contrast, in the abatacept trial, 17% of the patients were previously non-responders to TNF inhibitors, indicating a more therapy-resistant group.
With regard to trial design, the observed treatment duration in the etanercept trial (4 months) was shorter than in the adalimumab and abatacept trials (8 and 6 months, respectively). Because shorter trial duration means that there is a smaller chance of reaching the time-dependent outcome (disease flare), shorter treatment duration may result in better outcomes. On the other hand, the elimination time for etanercept (t1/2 70 h) is shorter than that for adalimumab and abatacept (both t1/2 ∼2 weeks), which might counteract the differences in trial duration. Owing to these differences, the estimate of comparative efficacy may be biased. An important conclusion from the comparisons in this network is that trials that are highly comparable with regard to design and case mix are needed in paediatric rheumatology.
Because these results could not be compared with similar analyses or with head-to-head trials including patients with JIA, the only feasible comparison is with data from observational studies performed in JIA. Extensive observational data on etanercept for polyarticular course JIA are available, all showing impressive effects both short and long term.33 ,34 Observational studies analysing adalimumab and abatacept in polyarticular course JIA are scarce.35 ,36 Adalimumab seems to be mainly preferred when uveitis is present.37 A small number of observational studies did compare the effectiveness of etanercept with that of infliximab in polyarticular course JIA. No differences in effectiveness were found; however, because of the chimeric structure of infliximab and the associated immunogenicity, infliximab was discontinued more often.38–40 Unfortunately, an indirect comparison between etanercept and infliximab could not be performed because of trial design dissimilarities. Parallel trials in the second network were highly comparable, and the indirect comparisons found anakinra, canakinumab and tocilizumab to be equally effective for systemic JIA. Unfortunately, no trials have investigated TNF inhibitors in systemic JIA only, and therefore the relative effect of TNF inhibitors compared with IL-1 and IL-6 blockers in systemic JIA could not be evaluated. Observational studies do indicate that etanercept is effective in some patients with systemic JIA.34 ,41 ,42 Many studies have found anakinra to be highly effective,43 ,44 but observational studies that evaluate canakinumab and tocilizumab in systemic JIA are still lacking.
Until conclusive differences are established, the choice of biological agent for JIA should mainly depend on drug availability, safety, practical reasons (such as interval of injections) and treatment costs. Further, more insight into features other than clinical characteristics, such as cytokine and genetic profiling, may contribute to this decision making. Focus could be shifted from treatment of the heterogeneous group of all JIA categories to tailored patient-specific care.
A recent review indicated that the National Institute for Health and Clinical Excellence prefers direct comparisons in decision making. Even in the absence of direct comparisons, key decisions are based on information from original trials, rather than available indirect comparisons.45 Head-to-head trials are still required, but, until conclusive direct evidence is established, decisions will have to be based on existing sources. In contrast with observational studies and meta-analyses, the indirect treatment comparison method allows the opportunity to preserve the strengths of randomisation. Thereby it is less likely that any differences between treatments observed through indirect comparisons are due to differences between patients that are unrelated to treatment effect.
The internal validity of indirect treatment comparison is highly dependent on (i) appropriate identification of studies, (ii) quality and internal validity of the included trials, and (iii) fulfilment of the similarity assumption.46 ,47 We identified all studies systematically, and, to minimise publication bias, checked trial registers; no additional trials that were completed could be identified. To our knowledge, all performed trials were identified. Trial quality was assessed, but varied greatly. Unfortunately, not all trials could be fully assessed, because data had to be extracted from conference abstracts.48 Bias may have been introduced, because these abstracts were not part of peer-reviewed publications. Earlier trials for registration commissioned by pharmaceutical industries gained the best scores, while more recent trials that evaluated treatment strategies for approved biological agents performed worst. The inferior trial quality of the TREAT trial was one of the reasons for excluding this trial from the indirect comparison. It is very likely that patients in the prednisone arm will develop signs of Cushing's syndrome, leading to unavoidable unblinding issues. The results of the withdrawal trials, although of high quality, should be placed in a different context. Instead of an estimation of the initial response to the treatment, the effect of treatment discontinuation has been researched. Better protection of flares after the primary response does not necessarily imply better initial treatment responses. It should be noted that, although the Jadad criteria assessed the most important aspects of quality of trial design, this does not guarantee that the trial was conducted perfectly.
The biggest challenge for this indirect comparison was fulfilling the similarity assumption, because of the heterogeneity of the disease and the scarcity of trials. Relying on clinical judgement, six trials were found to be suitable for indirect comparisons. The two networks consisted of methodologically identical trials with inclusion of patients with approximately the same disease duration and JIA categories, which were thought to be the most important confounders.34 ,49 Although classification criteria for juvenile arthritis differed between withdrawal trials (JRA and JIA), patients included were identical with regard to number of joints with arthritis.
Nevertheless, the included trials were not perfectly identical. Treatment duration varied between trials and is likely to have influenced the measured outcome. Furthermore, although the similarities predominated, differences between the withdrawal trials included co-medication used and previous treatment of some patients with biological agents. Because of the small number of trials and the limited number of patients included, it was unfeasible to further adjust our results using meta-regression, subgroup analysis or sensitivity analysis.
In conclusion, this review provides a comprehensive overview of the conducted trials that evaluated biological agents for treatment of JIA. Taking into account the differences between trials, this is the first study to carefully conclude that the short-term efficacy of etanercept, adalimumab and abatacept seems similar for polyarticular course JIA, and that of anakinra, canakinumab and tocilizumab seems similar for systemic JIA. Because of the observed differences between trials, more comparable trials and head-to-head trials directly comparing biological agents are urgently needed. For now, the paediatric rheumatologist must rely on these indirect comparisons, supplemented by observational data derived from cohort studies and safety, practical and financial arguments.
Handling editor Tore K Kvien
MHO and JA contributed equally.
Contributors All authors contributed substantially to the conception and design, or analysis and interpretation of data, drafting the article or revising it critically for important intellectual content, and provided final approval of the version to be published.
Competing interests MHO has received, outside the submitted work, travel grants from Pfizer, support for consultancy from Roche, and grants for thesis from Abbott, Novartis, Roche and Pfizer. LWAvS-S has received, outside the submitted work, consultancy fees from Pfizer, Roche and Novartis, grants from the Dutch Board of Health Insurances, Pfizer (formerly Wyeth), Abbott and the Dutch Arthritis Association, and travel expenses from Pfizer and Bristol-Myers Squibb.
Provenance and peer review Not commissioned; externally peer reviewed.