Article Text
PDF
Abstract
Background Recently, an American College of Rheumatology (ACR)/European League Against Rheumatism (EULAR) collaboration developed new classification criteria for rheumatoid arthritis (RA).
Objective To evaluate the diagnostic and discriminative ability of these new criteria compared with the 1987 ACR criteria and the Visser decision rule.
Methods 455 patients with early arthritis were studied. The diagnostic performance of the criteria was evaluated using methotrexate treatment within 1 year, expert opinion RA and erosive disease as ‘gold standards’. Erosive disease was defined as a 0–3 year change in radiographic score of ≥5.
Results The discriminative ability of the three criteria sets (2010 ACR/EULAR, 1987 ACR criteria and Visser algorithm) was similar with areas under the curve of 0.71–0.78 (‘gold standard’ methotrexate), 0.74–0.80 (gold standard expert opinion RA) and 0.63–0.67 (gold standard erosive disease after 3 years). The sensitivity of the 2010 ACR/EULAR criteria was highest with 0.85 (gold standard methotrexate). 86% of patients with RA and 51% of ‘non-RA’ patients according to the new criteria used methotrexate.
Conclusion The 2010 ACR/EULAR criteria were slightly more sensitive, but otherwise performed similarly to the older criteria. A high percentage of ‘non-RA’ patients used methotrexate, the gold standard for RA. The ability of the new criteria to identify patients with erosive disease was low, possibly owing to the effect of intensive treatment.
Statistics from Altmetric.com
Introduction
Early diagnosis and treatment of rheumatoid arthritis (RA) has become a major goal in rheumatology. Recently, an American College of Rheumatology (ACR)/European League Against Rheumatism (EULAR) collaboration developed new classification criteria for RA in order to facilitate the identification of patients who would benefit from early intervention.1 2 The former 1987 ACR criteria for RA were considered not sensitive enough in early disease.3 Consequently, other prediction rules or algorithms have been developed to predict which patients are at high risk of persistent/erosive disease or of progression from undifferentiated arthritis to RA.4 5 The aim of this study was (1) to validate the diagnostic performance of the 2010 ACR/EULAR classification criteria for RA, compared with the 1987 ACR criteria for RA and the prediction algorithm of Visser,5 in an independent early arthritis cohort (EAC) and (2) to examine the capacity of the new criteria to identify future erosive disease.
Methods
Study cohort
The study population comprised a cohort of patients with early arthritis from the EAC at the Jan van Breemen Institute in Amsterdam, The Netherlands. This ongoing cohort includes patients aged ≥18 years, with at least two swollen joints, symptom duration of <2 years and no prior disease-modifying antirheumatic drug (DMARD) treatment. Patients with osteoarthritis, crystal arthropathy, spondyloarthritis, systemic lupus erythematosus, Sjögren's syndrome and infectious arthritis are excluded. The study population consists mainly of patients with oligoarthritis or polyarthritis who would formerly be classified as having RA or undifferentiated arthritis. Data were used of patients who were included from 2000 onwards (to conform with the criteria development cohorts), had been followed up for at least 12 months and from whom baseline radiographs without typical RA erosions were available. Because of the absence of agreement on the definition of typical erosions we regarded erosion scores of >1, assessed with the Sharp–van der Heijde scoring (SHS) method, as typical RA erosions.6 Next, patients were scored according to the new classification criteria.2
The analysis of erosive disease is based on patients who had available radiographs at 0 and 3 years' follow-up. Radiographs of the hands, wrists and feet were scored with SHS by either one of two independent experienced observers who were aware of the chronological sequence of the radiographs. Interclass correlation between these two observers was 0.90 (95% CI 0.86 to 0.92).
Validation comparisons
We compared the 2010 ACR/EULAR criteria with the 1987 ACR criteria and the Visser algorithm. We were not able to include the van der Helm algorithm in the analysis because this algorithm was designed to be applied in patients who do not fulfil the 1987 ACR criteria at baseline, and of the remaining patients only 10 fulfilled the van der Helm criteria.4 Methotrexate treatment within 1 year after inclusion was used as ‘gold standard’ in accordance with the methods of the ACR/EULAR task force.1 In addition the performance of the 2010 ACR/EULAR criteria was calculated with expert opinion RA (defined as the diagnosis RA in the hospital diagnosis registry at 1 year after EAC inclusion) and erosive disease after 3-years' follow-up as alternative gold standards. Patients were considered to have erosive disease if the 3-year interval change in total SHS was ≥5.7
Statistical analysis
Two by two classification tables were generated to estimate sensitivity, specificity, positive and negative predictive value (PPV and NPV, respectively), and positive and negative likelihood ratio (LR+ and LR−, respectively) at the proposed cut-off values for the respective algorithms (2010 ACR/EULAR and the Visser algorithm ≥6, and 1987 ACR ≥4). To evaluate the capacity of the new criteria to discriminate between patients with and without methotrexate treatment within 1 year and the ability of the new criteria to identify erosive disease, receiver operating characteristic (ROC) curves with corresponding areas under the curve (AUC) were calculated.
Results
Patients
The patients had a mean age of 52 years, a median disease duration of 5.6 months and active disease as judged by the median Disease Activity Score (28-joint count) of 4.8 (table 1). Thirty-one patients (6.8%) had an erosion score of 1 at baseline.
Baseline characteristics of 455 patients with early arthritis, and radiographic progression data of 175 patients
Performance of the criteria
Three hundred and fifty-four (78%) patients had RA according to the gold standard (methotrexate use). Methotrexate treatment was given to 302 (86%) patients with RA and to 52 (51%) patients without RA according to the 2010 ACR/EULAR criteria. At baseline 78% fulfilled the 2010 ACR/EULAR criteria while 68% fulfilled the 1987 ACR criteria. Of 455 patients, 277 (61%) fulfilled both criteria sets, 76 (17%) patients fulfilled only the 2010 ACR/EULAR criteria and 22 (5%) fulfilled only the 1987 ACR criteria.
The AUC of the ROC curves show good discrimination for all three criteria sets between patients who did and did not receive methotrexate (table 2), with the highest value for the 2010 ACR/EULAR criteria. The sensitivity, specificity, PPV, NPV, LR+ and LR− of the three criteria sets at the proposed cut-off points for the three different gold standards are shown in table 2. The sensitivity of the 2010 ACR/EULAR criteria was higher than of the 1987 ACR criteria: 0.85 versus 0.76, respectively. The Visser algorithm cut-off points resulted in a lower sensitivity but higher specificity in our cohort. When we performed sensitivity analyses, and varied the cut-off point for typical RA erosions from >1 to >4, the results were similar (data not shown).
Performance of three sets of classification criteria for rheumatoid arthritis (RA) or for prediction of erosive disease
Of 455 patients, 310 (68%) had RA at 1 year according to expert opinion. Again the 2010 ACR/EULAR criteria had the highest score for sensitivity. The most common diagnoses after RA were oligoarthritis (n=44), psoriatic arthritis (n=31) and polyarthritis (n=27).
Prediction of erosive disease
Radiographs at 3 years' follow-up were available from 175 patients. In these patients median total SHS was 0 (IQR 0–0) at baseline and 0.0 (IQR 0–4) at 3 years (table 1). Twenty of 33 ‘non-RA’ patients according to the 2010 ACR/EULAR criteria (61%) in this subgroup used methotrexate within the first year.
The AUC of the ROC curves show a similar performance for the three criteria sets in the discrimination of patients with and without erosive disease (table 2). For all three criteria sets sensitivity was high (0.85–0.91) but specificity was low (0.17–0.41). When we performed sensitivity analyses, and varied the cut-off point for erosive disease from ≥3 to ≥8 increase in SHS points, the results remained similar (data not shown).
Discussion
In an independent EAC, we tested the performance of the 2010 ACR/EULAR and 1987 ACR classification criteria for RA and of the Visser decision rule for prediction of persistent/erosive disease. Apart from a higher frequency of anti-citrullinated protein antibody (52% vs 30%) in the presently studied EAC, there were no major differences with the EACs used for the derivation of the 2010 ACR/EULAR criteria.1 The three criteria sets had a similar ability to discriminate between RA or persistent arthritis and self-limiting arthritis. Sensitivity was highest with the 2010 ACR/EULAR criteria, and specificity was highest with the Visser rule. Recently, a study by van der Linden et al8 reported similar diagnostic and discriminative abilities of the 2010 ACR/EULAR criteria compared with the results of this study; however, their validation cohort was also used to develop the criteria. In this cohort the recommended cut-off points result in a high proportion of ‘false-positive’ patients—that is, patients classified as RA who did not receive methotrexate. If the new criteria were to be used as diagnostic instead of classification criteria, this could lead to unnecessary treatment with potentially toxic DMARDs.
One of the goals of the ACR/EULAR task force was to develop a criteria set that identifies patients at high risk of erosive disease.2 The discriminative ability of the 2010 ACR/EULAR criteria in this respect was similar to the Visser algorithm but slightly lower than the 1987 ACR criteria. The trend towards a lower radiographic progression that was observed in the group of ‘non-RA’ patients may be more the consequence of a high rate of methotrexate use in these patients than of a good prognosis itself in this group. The high rate of methotrexate treatment in the group of ‘non-RA’ patients suggests that the rheumatologists in our clinic had a more aggressive approach to early arthritis during the same period than the rheumatologists treating the cohorts that were used to derive the criteria. Obviously, this gold standard is not a static feature as rheumatologists have a tendency to treat earlier and more aggressively.
In conclusion the results suggest that, when applied to patients of this specific EAC with a high prevalence of RA, the new ACR/EULAR RA classification criteria are slightly more sensitive, and otherwise similar to, the older criteria. Since the 1987 ACR criteria set contains criteria that are highly uncommon in early disease, such as rheumatoid nodules, use of the new criteria is preferred.
Acknowledgments
The authors thank E de Wit-Taen, V van de Lugt en J Sterk for collecting patient data.
Footnotes
-
Funding Partly funded by the Dutch Arthritis Association.
-
Competing interests None.
-
Ethics approval This study was conducted with the approval of the local medical ethical committee.
-
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- Miscellaneous