Article Text

Download PDFPDF

Are American College of Rheumatology 50% response criteria superior to 20% criteria in distinguishing active aggressive treatment in rheumatoid arthritis clinical trials reported since 1997? A meta-analysis of discriminant capacities
  1. C P Chung1,
  2. J L Thompson2,5,
  3. G G Koch2,
  4. I Amara3,
  5. V Strand4,
  6. T Pincus1
  1. 1Division of Rheumatology and Immunology, Department of Medicine, Vanderbilt University Medical Center, Vanderbilt University, Nashville, Tennessee, USA
  2. 2Department of Biostatistics, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina, USA
  3. 3Quintiles, Durham, North Carolina
  4. 4Stanford University, Portola Valley, California, USA
  5. 5Department of Biostatistics, University of North Carolina, Chapel Hill, North Carolina, USA
  1. Correspondence to:
    T Pincus
    Division of Rheumatology and Immunology, Department of Medicine, Vanderbilt University Medical Center, 203 Oxford House, Box 5, Nashville, TN 37232-4500, USA; t.pincus{at}vanderbilt.edu

Abstract

Objective: To carry out a meta-analysis designed to compare the discriminant capacities of American College of Rheumatology 50% (ACR50) with 20% (ACR20) responses in clinical trials on rheumatoid arthritis reported after 1997 and to analyse whether ACR50 can be as informative as ACR20 in distinguishing active from control treatments in more recent trials.

Methods: Clinical trials on rheumatoid arthritis reported since 1997 were identified, which included aggressive combinations of disease-modifying antirheumatic drugs and glucocorticoids, as well as powerful new agents—leflunomide, etanercept, infliximab, anakinra, adalimumab, abatacept, tacrolimus and rituximab. A meta-analysis of ACR20 compared with ACR50 responses for 21 clinical trials was carried out on differences in proportions of responders for active and control treatments and corresponding odds ratios (ORs).

Results: In all but one clinical trial on rheumatoid arthritis published since 1997 with data available on ACR20 and ACR50, more than 50% of patients who were ACR20 responders among those randomised to active treatment were also ACR50 responders. This phenomenon was seen for control groups in 38% of trials, many of which included treatment with methotrexate. A meta-analysis of the clinical trials indicated a slight advantage to ACR50 for quantifying treatment comparisons, not significant for differences in proportions but significant for ORs.

Conclusion: ACR20 and ACR50 seem to be similar in distinguishing active from control treatments in clinical trials on rheumatoid arthritis reported since 1997. As ACR50 represents a considerably stronger clinical response, ACR50 may be a preferred end point for contemporary clinical trials on rheumatoid arthritis.

  • ACR20, American College of Rheumatology 20% response criteria
  • ACR50, American College of Rheumatology 50% response criteria
  • DMARD, disease-modifying antirheumatic drug

Statistics from Altmetric.com

The American College of Rheumatology (ACR) core dataset1–3 has been a major advance in randomised controlled clinical trials on rheumatoid arthritis. Improvement criteria have been established as 20%, 50% and 70% responses at end point compared with baseline, and termed ACR20, ACR50 and ACR70, requiring improvement in both swollen and tender joint counts, as well as three of the other five core dataset measures.4 In a 1998 report, ACR20 responses were found to have greater discriminant capacity to distinguish active from control treatment than ACR50 or ACR70 responses.5

Although ACR20 responses may have significant capacities to distinguish active from control treatment, such levels generally do not represent an optimal clinical improvement.6 Indeed, the reported superiority of ACR20 compared with ACR50 or ACR705 suggests that most treatments for rheumatoid arthritis reported before 1998 had limited efficacies, albeit statistically significantly greater than seen for placebo or control treatment. In many chronic diseases, such as hypertension or hyperlipidaemia, clinicians seek improvement at >20% levels.

Since the 1998 report, more aggressive treatment strategies have been introduced, including earlier use of disease-modifying antirheumatic drugs (DMARDs),7,8 with methotrexate as an “anchor” DMARD, often in combinations in a preventive strategy9,10 with new DMARDs and biological agents. Reports of clinical trials on methotrexate in combination with other DMARDs,11–13 and new agents such as monotherapy or in combination with methotrexate or other DMARDs, including leflunomide,14,15 etanercept,16,17 infliximab,18,19 anakinra,20,21 adalimumab,22,23 tacrolimus,24 abatacept25 and rituximab,26 indicate higher levels of both ACR20 and ACR50 responses than were seen in clinical trials included in the 1998 analysis.5

These findings raised consideration of a reassessment of ACR50 versus ACR20 as an end point in contemporary clinical trials on rheumatoid arthritis. Therefore, we identified all 21 clinical trials published between 1997 and 2004 in which data were available on both ACR20 and ACR50 responses for comparisons of active with control treatment. In this report, we present two meta-analyses to compare ACR20 with ACR50 responses, one for differences between treatments in the proportions of responders and the other for odds ratios (ORs).

METHODS

Reports of clinical trials on rheumatoid arthritis published since 1997 were reviewed to evaluate responses to combination therapies,11–13 leflunomide,14,27–30 etanercept,16,31–33 infliximab,19,34 adalimumab,23,35,36 anakinra,21 tacrolimus,24,37 abatacept25 and rituximab.26 Trials were included only if data on the proportion of patients who met both ACR20 and ACR50 response criteria with each treatment were reported. Therefore, published reports of several important clinical trials38–40 could not be included. Only one published report for each clinical trial was included. If more than one report was available, the report with a period of observation closest to 52 weeks was chosen. Overall, 21 trials were included in the analyses, with the primary treatment comparisons illustrated in table 1.

Table 1

 Data for the main treatment comparisons of interest

The proportions of patients who met ACR20 or ACR50 response criteria were identified for each treatment. A meta-analysis of the clinical trials was carried out to compare ACR20 and ACR50 as criteria for assessing active versus control treatments for change from baseline to end point. (The term “control” rather than “placebo” is used, as patients generally took a non-steroidal anti-inflammatory drug and or glucocorticoid, as well as possibly methotrexate, albeit after selection for incomplete responses to methotrexate.)

The term “meta-analysis” is used to describe an integrated quantitative comparison of the findings of many clinical trials in one analysis. This approach to meta-analysis differed from most common meta-analyses of clinical trials, which could be directed to analyse the efficacy of active versus control treatments. Our meta-analysis was directed to compare two outcome criteria for their discriminant capacities to detect differences between active and control treatments, as reported in the medical literature.

Meta-analyses were carried out for two types of comparisons: differences between active and control treatments in proportions responding, and in ORs for proportions responding. The extent to which the difference between treatments for the proportion of patients who met ACR50 was larger than the corresponding difference in the proportion who met ACR20 was described for each study by the corresponding “difference of differences” (ACR50−ACR20). If more than one treatment comparison was reported, the comparison with the most significant difference in differences versus zero (ie, smallest two-sided p value) was included in the analyses. This comparison was not the primary comparison for four clinical trials, as indicated in table 1, as differences between ACR50 and ACR20 (independent of the level of significance of the corresponding treatment differences) were more significant for the comparison that was analysed. The significance of the extent to which one of the two end points consistently had larger differences between treatments for responding proportions was evaluated across studies, using the Wilcoxon signed ranks test on the respective differences of differences. A Hodges–Lehmann 95% confidence interval (CI) was used to describe the magnitude of the median difference between end points in proportions responding across studies.41

ORs for the comparisons between active and control treatments were also analysed from the proportions of patients responding at ACR50 and ACR20 levels. The ratios of these two ORs were analysed to compare the end points for each study. Studies with more than one treatment comparison were represented by the ratio of ORs for which the logarithm was most significantly different from zero (ie, smallest two-sided p value). The significance of the extent to which larger ORs were obtained for treatment comparisons according to one of the two end points was evaluated using the Wilcoxon signed ranks test on the respective logarithms of the ratios of ORs. The anti-logarithms of the corresponding Hodges–Lehmann 95% CI were used to describe the magnitude of the median ratio of ORs across studies.41

RESULTS

An initial review was conducted of clinical trials included in the 1998 report, which indicated superior discrimination between active and control treatment according to comparison of ACR20 with ACR50.5 Data on the primary treatment comparisons of interest in each trial are shown in table 1. Table 1 also indicates the four trials in which the comparison of ACR20 with ACR50 shown in the figures differed from the primary comparison, as these trials were more significant for ACR20 versus ACR50. Results from the three trials conducted by the Cooperative Systematic Studies of the Rheumatic Diseases indicated ACR20 response in 40% of patients, consisting of 31% of patients with ACR20 but no ACR50 response and only 9% with ACR50 responses. Therefore, almost 75% of the patients who responded at the ACR20 level did not respond at the ACR50 level.

Analyses of more recent clinical trials on combination therapies suggest that most patients who met ACR20 also met ACR50 response criteria. In the trial conducted by Boers et al,11 72% of patients who received glucocorticoids and triple therapy met ACR20 criteria and 49% met ACR50 criteria at 1 year. Therefore, 68% of ACR20 responders were also ACR50 responders. Similarly, in the study by Möttönen et al,12 78% of patients receiving combination therapy met ACR20 and 71% met ACR50 criteria; 91% of ACR20 responders were also ACR50 responders, the highest proportion of ACR20 responders who were also ACR50 responders of any trial (including those with biological agents) published to date.

In all the other clinical trials except one,37 more than 50% (51–82%) of patients who received active treatment and met ACR20 criteria also met ACR50 criteria. Furthermore, in 8 of the 21 studies (38%), more than 50% of patients who met ACR20 criteria in the control groups of the primary treatment comparisons also met ACR50 criteria, possibly partly owing to treatment of patients in many control groups with methotrexate.

Figure 1 shows a meta-analysis to compare treatment differences for the proportions of patients who met ACR50 versus their counterparts meeting ACR20 through the most significant difference of differences for each trial. Overall, 12 reports indicated greater differences between active and control treatments according to ACR20 versus ACR50 criteria, four of which were significant. Four reports indicated virtually identical results according to both response criteria and five reports indicated a larger treatment difference of response according to ACR50 compared to ACR20 criteria; two of these differences were significant (fig 1). The meta-analysis comparison of all studies indicated a slight advantage to ACR20, although the Hodges–Lehmann 95% CI for ACR50 versus ACR20 included the zero point, indicating that it was not significant (fig 1).

Figure 1

 Meta-analysis of ACR50 versus ACR20 differences for proportions of patients meeting each end point with active versus control treatments in 21 clinical trials. The treatment comparison with the difference of differences that is most significantly different from zero is shown for each trial. The Hodges–Lehmann confidence interval (CI) from an analysis of combined studies shows that the median difference of differences is not significantly different from zero, showing no major advantage of ACR20 or ACR50. ABAT, abatacept; ADA, adalimumab; ANAK, anakinra; CYCLO, cyclosporine; DMARD, disease-modifying antirheumatic drug; ETAN, etanercept; HCQ, hydroxychloroquine; INFL, infliximab; LEF, leflunomide; MTX, methotrexate; PBO, placebo; PRED, prednisone; RTX, rituximab; SSZ, sulphasalazine; TAC, tacrolimus.

In a meta-analysis of ACR50 versus ACR20 according to the most significant ratio of active versus control ORs for each trial, CIs in general were much wider than their counterparts for differences of differences (fig 2). Overall, seven studies favoured ACR20, generally the same studies with this result for differences between proportions, although none were significant in an OR analysis. By contrast, 14 studies favoured ACR50, three of which were significant. The Hodges–Lehmann 95% CI for this meta-analysis indicated a significant (although small) advantage to ACR50 versus ACR20 responses according to ORs (fig 2).

Figure 2

 Meta-analysis of ACR50 versus ACR20 ratios of odds ratios (OR) for meeting each end point with active versus control treatments in 21 clinical trials. The treatment comparison for which the logarithm of the ratio of ratios is most significantly different from zero is shown for each trial. The Hodges–Lehmann confidence interval (CI) from the analysis of the combined studies shows that the median ratio of ORs is significantly >1, indicating a slight advantage of ACR50 compared with ACR20. ABAT, abatacept; ADA, adalimumab; ANAK, anakinra; CYCLO, cyclosporine; DMARD, disease-modifying antirheumatic drug; ETAN, etanercept; HCQ, hydroxychloroquine; INFL, infliximab; LEF, leflunomide; MTX, methotrexate; PBO, placebo; PRED, prednisone; RTX, rituximab; SSZ, sulphasalazine; TAC, tacrolimus.

DISCUSSION

These data confirm the validity of ACR20 as an optimal discriminator to distinguish active versus control treatment in clinical trials conducted before 1997, in agreement with the published report.5 In trials conducted since 1997, however, results according to ACR50 seem similar to those according to ACR20. In the meta-analyses, treatment differences for ACR50 proportions did not differ significantly from their ACR20 counterparts, whereas ratios of ORs slightly favoured ACR50, with a significant advantage. Both types of meta-analysis indicated agreement in most clinical trials, although point estimates in 7 of the 21 trials in table 1 differ in the end point they favour according to the two methods. However, the 95% CIs overlap the value for no difference in all but one of these cases in both figures, illustrating both the value of CIs in accounting for the random variability of point estimates and the possible value of analysing the discriminant capacity of the two end points using two different methods.

As noted in the Methods section, the meta-analysis reported here is in some ways similar to a traditional meta-analysis, as it integrates findings in many clinical trials into one quantitative analysis. However, the goal in our studies was to compare discriminant capacities of results reported according to ACR20 and ACR50 criteria to detect differences between active and control treatments, rather than to analyse the efficacy of a particular treatment versus a control in many different clinical trials. Furthermore, this review was directed to compare responses reported as ACR20 versus ACR50, as described in the medical literature, without any comment on the implications regarding the treatments under study.

These considerations led us to not deal with matters that might be included in a meta-analysis on a given treatment, such as differences in patient populations, duration of studies, methods to account for missing data, or doses of methotrexate or no methotrexate in a control group. Any influence of these matters on results is arguably similar for ACR20 and ACR50 in each trial, and thereby seems minimal for a comparison of these two criteria. Moreover, heterogeneity among trials was managed as random variability, using the methods for obtaining overall results. Further analyses may provide additional information on the optimal methods to assess clinical responses in trials on rheumatoid arthritis, but could require access to the original data, which were not available to the authors. Other approaches to analysing comparisons between active versus control treatments in clinical trials on rheumatoid arthritis, including the Disease Activity Score42 and indices that include only patient measures,43 were also not dealt with here.

The meta-analysis of 21 clinical trials conducted on patients with rheumatoid arthritis since 1997 indicated similar discriminant capacities of ACR20 and ACR50 responses. The evidence that active versus control treatments can be distinguished according to ACR50, and according to ACR20, seems to attest the superiority of newer therapeutic approaches and agents versus those reported before 1998. The effective discriminant capacity of ACR50 versus ACR20 was as likely in trials on early and aggressive combinations of DMARDs as in trials on biological agents, suggesting that an aggressive approach may be as important as a specific new agent in improving outcomes in rheumatoid arthritis. Further analyses using different methods of calculating responses will contribute to improved methods to analyse comparisons between active and control treatments in clinical trials on rheumatoid arthritis. On the basis of the meta-analyses reported here, it seems reasonable to conclude that ACR50 may be a preferred end point, as it is a considerably more desirable target for people who have rheumatoid arthritis.

REFERENCES

Footnotes

  • Published Online First 27 February 2006

  • Funding: This work was supported in part by grants from The Arthritis Foundation, Abbott Immunology, and the Centocor Health Outcomes in Rheumatic Diseases (CHORD) Fellowship Program.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.