Dear Editor,
Radner and colleagues report a systematic literature search analysing
the numerous articles and conference proceedings which examined the
performance of the 2010 ACR/EULAR classification criteria for rheumatoid
arthritis (RA) (1). The comprehensive evaluation identified if the 2010
criteria were correctly applied as suggested in the original publication,
explored the performance of the criteria according to different methods to
assess the criteria components and with use of different reference
standards, and finally, directly compared the results obtained upon
classification when using the 2010 or the 1987 criteria.
The overall
sensitivity of the criteria was 82% and overall specificity was 61%, when
applied to the intended target population. Eight studies and five meeting
abstracts directly compared 1987 and 2010 criteria using different
reference standards within different target populations. When excluding
patients with other diagnosis, the 2010 ACR/ EULAR criteria demonstrated
almost 21% higher sensitivity compared with 1987 ACR criteria, whereas
specificity was 16% lower. Therefore, Radner et al. conclude that the 2010
criteria are more sensitive than the 1987 criteria at the cost of a slight
decrement in specificity, which might increase the possibility that a few
non-RA patients are classified as RA patients and, for example, entered
into clinical trials (1). Another recent systematic literature review and
a meta-analysis including 6 full papers and 4 abstracts reported identical
performance stating that the new classification criteria have good
sensitivity, lower specificity and an overall moderate diagnostic accuracy
(2). Sakellariou et al. interpret the results as confirmation of the value
of the criteria for classification but not for diagnosis (2).
Indeed, the first prospective study of consecutive patients seen in
routine care of an outpatient clinic of a university rheumatology centre
using the doctor's diagnosis as the gold standard, different from using
medication start, found that the sensitivity and specificity of the 2010
criteria in classifying RA were 97% and 55%, respectively, compared with
the 1987 RA criteria which were 93% and 76%, respectively (3). More
specifically, 66.7% of systemic lupus erythematosus patients, 50% of
osteoarthritis, 37.5% of psoriatic arthritis and 27.2% of others fulfilled
the new criteria and could have been incorrectly classified as RA. Thus,
testing the criteria for the first time in a broad spectrum of
rheumatological diseases seen in routine rheumatology care confirm the
concern that the poor specificity may lead to over- and misdiagnosis of
patients with RA, leading to inappropriate medication use (4, 5).
Vunkemann and van de Laar reviewing the performance of the criteria very
recently concluded:"Especially, when the classification criteria are used
as diagnostic criteria this carries the risk of overtreatment. It remains
to be determined whether or not the new criteria when used to diagnose and
treat patients provide an acceptable balance between efficacy and safety
and in these days also of major importance, cost-effectiveness"(6). In
keeping with overclassification, several evaluation studies revealed that
patients classified as RA according to the 2010 criteria are more likely
to achieve drug-free remission than those who meet the 1987 criteria (7,
8, 9). Moreover, an ongoing prospective study of early arthritis
demonstrated that 51% of 2010 criteria "non-RA" patients compared to 86%
of patients with RA were treated with methotrexat (MTX) in the first year,
suggesting that the rheumatologists in their clinic had a more aggressive
approach to early arthritis during the same period than the
rheumatologists treating the cohorts that were used to derive the criteria
(10). Obviously, MTX is neither a "gold standard" for RA nor a static
feature, as rheumatologists have a tendency to treat earlier and more
aggressively. In addition, MTX and DMARD medications are also prescribed
for other chronic inflammatory diseases, such as psoriatic arthritis and
peripheral spondyloarthritis. Most importantly, classification criteria
serve as a tool to arrive at homogeneous groups of patients with
comparable features to make data obtained by different researchers at
different places comparable why they should have a high specificity
(preferably close to 100%), in order to prevent misclassification and
inclusion of patients who do not have the disease (11). Also a balance of
sensitivity and specificity is required in validation of criteria sets for
use in clinical trials and epidemiologic studies (12).
Obviously, these
requirements are hardly met if in the appropriate intended population the
area under curve for receiver operating characteristic curves are rather
weak between 0.72 and 0.78 indicating misclassification in 22% to 28% and
limited accuracy to separate RA from other early arthritides (10, 13, 14).
Overall, the question arises whether the loss of specificity is a price
worth paying to use the criteria for classification in clinical trials.
In conclusion, the good news of better sensitivity is considerably
limited by the loss of specificity and related risk of overtreatment of
patients with self-limiting disease as RA with potentially toxic agents,
even considering that poor recognition and inadequate intervention in the
earliest phases of inflammatory arthritis may occur more often. The
improvement of the accuracy of diagnosis and classification of early and
very early RA remain a continuous challenge. Recently we reviewed
proposals and suggestions how to overcome the problems and limitations of
the 2010 criteria by clinical practice and future research (4):
1) the rheumatologist as the expert strikes a balance between possible or
probable RA, depending on the level of confidence (15),
2) the rheumatologist uses a diagnostic certainty scale at baseline (0 to
100 visual analog scale) (16),
3) use of the prediction rule developed by van der Helm-van Mil et al.
(17, 18) to estimate the chance of progression to RA in individual
patients presenting with undifferentiated arthritis,
4) use of imaging techniques (sonography, MRI) to identify erosions
earlier (19, 20, 21),
5) testing the discriminative value of HLA-B27 and diagnostic programs for
reactive arthritis (22, 23),
6) testing likelihood ratios for diagnostic decision-making based on the
Bayesian approach (24),
7) use of automated, multiplex biomarker assay testing for autoantibodies,
cytokines, and bone-turnover products (25).
Finally, future research should focus on validating the 2010 criteria in
terms of important long-term outcomes in RA such as radiological damage,
disability and mortality (5). In this regard, most recently a study of
early arthritis patients evaluated the ability of the 2010 ACR/ EULAR and
the 1987 ACR classification criteria to predict radiographic progression
after 10 years of follow-up (27). The data suggests that both
classification criteria predict poorly erosive disease in patients with
early RA. The discriminative power of the 2010 criteria is only slightly
better than that of the 1987 criteria with area under the curve of 0.72
and 0.65, respectively. Another most recent study reported that the 2010
criteria appear to be as efficient as the 1987 criteria in identifying
increased risk of mortality but the 2010 criteria identify a greater
proportion of at-risk patients soon after their first presentation to
health care with a hazard ratio of 1.35 compared to 1.24 (28).
References
1 Radner H, Radner H, Neogi T, Smolen JS, et al. Ann Rheum Dis Published
Online First: [April 16, 2013] doi:10.1136/ annrheumdis-2013-203284
2 Sakellariou G, Scire` CA, Zambon A et al. Performance of the 2010
Classification Criteria for Rheumatoid Arthritis: A Systematic Literature
Review and a Meta-Analysis. PLoS ONE 2013;8:e56528.
3 Kennish L, Labitigan M, Budoff S et al. Utility of the new rheumatoid
arthritis 2010 ACR/EULAR classification criteria in routine clinical care.
BMJ Open 2012;2:e001117.
4 Zeidler H. The need to better classify and diagnose early and very early
rheumatoid arthritis. J Rheumatol 2012;39:212-7.
5 Humphreys JH, Symmons DP. Postpublication validation of the 2010
American College of Rheumatology/European League Against Rheumatism
classification criteria for rheumatoid arthritis: where do we stand? Curr
Opin Rheumatol 2013;25:157-63.
6 Vonkeman HE, van de Laar MA. The new European League Against
Rheumatism/American College of Rheumatology diagnostic criteria for
rheumatoid arthritis: how are they performing? Curr Opin Rheumatol
2013;25:354-9.
7 Cader MZ, Filer A, Hazlehurst J, et al. Performance of the 2010
ACR/EULAR criteria for rheumatoid arthritis: comparison with 1987 ACR
criteria in a very early synovitis cohort. Ann Rheum Dis 2011;70:949-55.
8 De Hair MJ, Lehmann KA, van de Sande MG, et al. The clinical picture of
rheumatoid arthritis according to the 2010 American College of
Rheumatology/European League Against Rheumatism criteria: is this still
the same disease? Arthritis Rheum 2012;64:389-93.
9 Krabben A, Huizinga TW, van der Helm-van Mil AH. Undifferentiated
arthritis characteristics and outcomes when applying the 2010 and 1987
criteria for rheumatoid arthritis. Ann Rheum Dis 2012;71:238-241.
10 Britsemmer K, Ursum J, Gerritsen M, van Tuyl LH et al. Validation of
the 2010 ACR/EULAR classification criteria for rheumatoid arthritis:
slight improvement over the 1987 ACR criteria. Ann Rheum Dis 2011;70:1468-
70.
11 van der Helm-van Mil AH, Huizinga TWJ. The 2010 ACR/EULAR criteria for
rheumatoid arthritis: do they affect the classification or diagnosis of
rheumatoid arthritis? Ann Rheum Dis 2012;71:1596-98.
12 Johnson SR, Goek ON, Singh-Grewal D et al. Classification criteria in
rheumatic diseases: A review of methodologic properties. Arthritis Rheum
2007;57:1119-33.
13 van der Linden MP, Knevel R, Huizinga TW et al. Classification of
rheumatoid arthritis: Comparison of the 1987 American College of
Rheumatology criteria and the 2010 American College of
Rheumatology/European League Against Rheumatism criteria. Arthritis Rheum
2011;63:37-42.
14 Varache S, Cornec D, Morvan J et al. Diagnostic accuracy of ACR/EULAR
2010 criteria for rheumatoid arthritis in a 2-year cohort. J Rheumatol
2011;38:1250-7.
15 Benhamou M, Rincheval N, Roy C et al. The gap between practice and
guidelines in the choice of first-line disease modifying antirheumatic
drug in early rheumatoid arthritis: Results from the ESPOIR cohort. J
Rheumatol 2009;36:934-42.
16 Morvan J, Berthelot JM, Devauchelle-Pensec V et al. Changes over time
in the diagnosis of rheumatoid arthritis in a 10-year cohort. J Rheumatol
2009;36:2428-34
17 van der Helm-van Mil AH, Detert J, le Cessie S et al. Validation of a
prediction rule for disease outcome in patients with recent-onset
undifferentiated arthritis: Moving toward individualized treatment
decision-making. Arthritis Rheum 2008;58:2241-7.
18 Huizinga TW, van der Helm-van Mil A. Quantitative approach to early
rheumatoid arthritis. Bull NYU Hosp Joint Dis 2011;69:116-21.
19 Freeston JE, Wakefield RJ, Conaghan PG et al. A diagnostic algorithm
for persistence of very early inflammatory arthritis: The utility of power
Doppler ultrasound when added to conventional assessment tools. Ann Rheum
Dis 2010;69:417-9.
20 Boutry N, do Carmo CC, Flipo RM et al. Early rheumatoid arthritis and
its differentiation from other joint abnormalities. Eur J Radiol
2009;71:217-24.
21 Filer A, de Pablo P, Allen G et al. Utility of ultrasound joint counts
in the prediction of rheumatoid arthritis in patients with very early
synovitis. Ann Rheum Dis 2011;70:500-7.
22 Hülsemann JL, Zeidler H. Diagnostic evaluation of classification
criteria for rheumatoid arthritis and reactive arthritis in an early
synovitis outpatient clinic. Ann Rheum Dis 1999;58:278-80.
23 Soderlin MK, Borjesson O, Kautiainen H et al. Annual incidence of
inflammatory joint diseases in a population based study in southern
Sweden. Ann Rheum Dis 2002;61:911-5.
24 Corrao S, Calvo L, Licata G. The new criteria for classification of
rheumatoid arthritis: What we need to know for clinical practice. Eur J
Intern Med 2011;22:217-9.
25 Chandra PE, Sokolove J, Hipp BG et al. Novel multiplex technology for
diagnostic characterization of rheumatoid arthritis. Arthritis Res Ther
2011;13:R102.
26 M?kinen H, Kaarela K, Huhtala H et al. Do the 2010 ACR/EULAR or ACR
1987 classification criteria predict erosive disease in early arthritis?
Ann Rheum Dis 2013;72:745-7.
27 Humphreys JH, Verstappen SM, Hyrich KL et al. 2010 ACR/EULAR
classification criteria for rheumatoid arthritis predict increased
mortality in patients with early arthritis: results from the Norfolk
Arthritis Register. Rheumatology (Oxford) 2013 Mar 5. [Epub ahead of
print] doi:10.1093/rheumatology/ket113.
Conflict of Interest:
None declared
Dear Editor,
Radner and colleagues report a systematic literature search analysing the numerous articles and conference proceedings which examined the performance of the 2010 ACR/EULAR classification criteria for rheumatoid arthritis (RA) (1). The comprehensive evaluation identified if the 2010 criteria were correctly applied as suggested in the original publication, explored the performance of the criteria according to different methods to assess the criteria components and with use of different reference standards, and finally, directly compared the results obtained upon classification when using the 2010 or the 1987 criteria.
The overall sensitivity of the criteria was 82% and overall specificity was 61%, when applied to the intended target population. Eight studies and five meeting abstracts directly compared 1987 and 2010 criteria using different reference standards within different target populations. When excluding patients with other diagnosis, the 2010 ACR/ EULAR criteria demonstrated almost 21% higher sensitivity compared with 1987 ACR criteria, whereas specificity was 16% lower. Therefore, Radner et al. conclude that the 2010 criteria are more sensitive than the 1987 criteria at the cost of a slight decrement in specificity, which might increase the possibility that a few non-RA patients are classified as RA patients and, for example, entered into clinical trials (1). Another recent systematic literature review and a meta-analysis including 6 full papers and 4 abstracts reported identical performance stating that the new classification criteria have good sensitivity, lower specificity and an overall moderate diagnostic accuracy (2). Sakellariou et al. interpret the results as confirmation of the value of the criteria for classification but not for diagnosis (2).
Indeed, the first prospective study of consecutive patients seen in routine care of an outpatient clinic of a university rheumatology centre using the doctor's diagnosis as the gold standard, different from using medication start, found that the sensitivity and specificity of the 2010 criteria in classifying RA were 97% and 55%, respectively, compared with the 1987 RA criteria which were 93% and 76%, respectively (3). More specifically, 66.7% of systemic lupus erythematosus patients, 50% of osteoarthritis, 37.5% of psoriatic arthritis and 27.2% of others fulfilled the new criteria and could have been incorrectly classified as RA. Thus, testing the criteria for the first time in a broad spectrum of rheumatological diseases seen in routine rheumatology care confirm the concern that the poor specificity may lead to over- and misdiagnosis of patients with RA, leading to inappropriate medication use (4, 5). Vunkemann and van de Laar reviewing the performance of the criteria very recently concluded:"Especially, when the classification criteria are used as diagnostic criteria this carries the risk of overtreatment. It remains to be determined whether or not the new criteria when used to diagnose and treat patients provide an acceptable balance between efficacy and safety and in these days also of major importance, cost-effectiveness"(6). In keeping with overclassification, several evaluation studies revealed that patients classified as RA according to the 2010 criteria are more likely to achieve drug-free remission than those who meet the 1987 criteria (7, 8, 9). Moreover, an ongoing prospective study of early arthritis demonstrated that 51% of 2010 criteria "non-RA" patients compared to 86% of patients with RA were treated with methotrexat (MTX) in the first year, suggesting that the rheumatologists in their clinic had a more aggressive approach to early arthritis during the same period than the rheumatologists treating the cohorts that were used to derive the criteria (10). Obviously, MTX is neither a "gold standard" for RA nor a static feature, as rheumatologists have a tendency to treat earlier and more aggressively. In addition, MTX and DMARD medications are also prescribed for other chronic inflammatory diseases, such as psoriatic arthritis and peripheral spondyloarthritis. Most importantly, classification criteria serve as a tool to arrive at homogeneous groups of patients with comparable features to make data obtained by different researchers at different places comparable why they should have a high specificity (preferably close to 100%), in order to prevent misclassification and inclusion of patients who do not have the disease (11). Also a balance of sensitivity and specificity is required in validation of criteria sets for use in clinical trials and epidemiologic studies (12).
Obviously, these requirements are hardly met if in the appropriate intended population the area under curve for receiver operating characteristic curves are rather weak between 0.72 and 0.78 indicating misclassification in 22% to 28% and limited accuracy to separate RA from other early arthritides (10, 13, 14). Overall, the question arises whether the loss of specificity is a price worth paying to use the criteria for classification in clinical trials.
In conclusion, the good news of better sensitivity is considerably limited by the loss of specificity and related risk of overtreatment of patients with self-limiting disease as RA with potentially toxic agents, even considering that poor recognition and inadequate intervention in the earliest phases of inflammatory arthritis may occur more often. The improvement of the accuracy of diagnosis and classification of early and very early RA remain a continuous challenge. Recently we reviewed proposals and suggestions how to overcome the problems and limitations of the 2010 criteria by clinical practice and future research (4):
1) the rheumatologist as the expert strikes a balance between possible or probable RA, depending on the level of confidence (15),
2) the rheumatologist uses a diagnostic certainty scale at baseline (0 to 100 visual analog scale) (16),
3) use of the prediction rule developed by van der Helm-van Mil et al. (17, 18) to estimate the chance of progression to RA in individual patients presenting with undifferentiated arthritis,
4) use of imaging techniques (sonography, MRI) to identify erosions earlier (19, 20, 21),
5) testing the discriminative value of HLA-B27 and diagnostic programs for reactive arthritis (22, 23),
6) testing likelihood ratios for diagnostic decision-making based on the Bayesian approach (24),
7) use of automated, multiplex biomarker assay testing for autoantibodies, cytokines, and bone-turnover products (25). Finally, future research should focus on validating the 2010 criteria in terms of important long-term outcomes in RA such as radiological damage, disability and mortality (5). In this regard, most recently a study of early arthritis patients evaluated the ability of the 2010 ACR/ EULAR and the 1987 ACR classification criteria to predict radiographic progression after 10 years of follow-up (27). The data suggests that both classification criteria predict poorly erosive disease in patients with early RA. The discriminative power of the 2010 criteria is only slightly better than that of the 1987 criteria with area under the curve of 0.72 and 0.65, respectively. Another most recent study reported that the 2010 criteria appear to be as efficient as the 1987 criteria in identifying increased risk of mortality but the 2010 criteria identify a greater proportion of at-risk patients soon after their first presentation to health care with a hazard ratio of 1.35 compared to 1.24 (28).
References
1 Radner H, Radner H, Neogi T, Smolen JS, et al. Ann Rheum Dis Published Online First: [April 16, 2013] doi:10.1136/ annrheumdis-2013-203284
2 Sakellariou G, Scire` CA, Zambon A et al. Performance of the 2010 Classification Criteria for Rheumatoid Arthritis: A Systematic Literature Review and a Meta-Analysis. PLoS ONE 2013;8:e56528.
3 Kennish L, Labitigan M, Budoff S et al. Utility of the new rheumatoid arthritis 2010 ACR/EULAR classification criteria in routine clinical care. BMJ Open 2012;2:e001117.
4 Zeidler H. The need to better classify and diagnose early and very early rheumatoid arthritis. J Rheumatol 2012;39:212-7.
5 Humphreys JH, Symmons DP. Postpublication validation of the 2010 American College of Rheumatology/European League Against Rheumatism classification criteria for rheumatoid arthritis: where do we stand? Curr Opin Rheumatol 2013;25:157-63.
6 Vonkeman HE, van de Laar MA. The new European League Against Rheumatism/American College of Rheumatology diagnostic criteria for rheumatoid arthritis: how are they performing? Curr Opin Rheumatol 2013;25:354-9.
7 Cader MZ, Filer A, Hazlehurst J, et al. Performance of the 2010 ACR/EULAR criteria for rheumatoid arthritis: comparison with 1987 ACR criteria in a very early synovitis cohort. Ann Rheum Dis 2011;70:949-55.
8 De Hair MJ, Lehmann KA, van de Sande MG, et al. The clinical picture of rheumatoid arthritis according to the 2010 American College of Rheumatology/European League Against Rheumatism criteria: is this still the same disease? Arthritis Rheum 2012;64:389-93.
9 Krabben A, Huizinga TW, van der Helm-van Mil AH. Undifferentiated arthritis characteristics and outcomes when applying the 2010 and 1987 criteria for rheumatoid arthritis. Ann Rheum Dis 2012;71:238-241.
10 Britsemmer K, Ursum J, Gerritsen M, van Tuyl LH et al. Validation of the 2010 ACR/EULAR classification criteria for rheumatoid arthritis: slight improvement over the 1987 ACR criteria. Ann Rheum Dis 2011;70:1468- 70.
11 van der Helm-van Mil AH, Huizinga TWJ. The 2010 ACR/EULAR criteria for rheumatoid arthritis: do they affect the classification or diagnosis of rheumatoid arthritis? Ann Rheum Dis 2012;71:1596-98.
12 Johnson SR, Goek ON, Singh-Grewal D et al. Classification criteria in rheumatic diseases: A review of methodologic properties. Arthritis Rheum 2007;57:1119-33.
13 van der Linden MP, Knevel R, Huizinga TW et al. Classification of rheumatoid arthritis: Comparison of the 1987 American College of Rheumatology criteria and the 2010 American College of Rheumatology/European League Against Rheumatism criteria. Arthritis Rheum 2011;63:37-42.
14 Varache S, Cornec D, Morvan J et al. Diagnostic accuracy of ACR/EULAR 2010 criteria for rheumatoid arthritis in a 2-year cohort. J Rheumatol 2011;38:1250-7.
15 Benhamou M, Rincheval N, Roy C et al. The gap between practice and guidelines in the choice of first-line disease modifying antirheumatic drug in early rheumatoid arthritis: Results from the ESPOIR cohort. J Rheumatol 2009;36:934-42.
16 Morvan J, Berthelot JM, Devauchelle-Pensec V et al. Changes over time in the diagnosis of rheumatoid arthritis in a 10-year cohort. J Rheumatol 2009;36:2428-34
17 van der Helm-van Mil AH, Detert J, le Cessie S et al. Validation of a prediction rule for disease outcome in patients with recent-onset undifferentiated arthritis: Moving toward individualized treatment decision-making. Arthritis Rheum 2008;58:2241-7.
18 Huizinga TW, van der Helm-van Mil A. Quantitative approach to early rheumatoid arthritis. Bull NYU Hosp Joint Dis 2011;69:116-21.
19 Freeston JE, Wakefield RJ, Conaghan PG et al. A diagnostic algorithm for persistence of very early inflammatory arthritis: The utility of power Doppler ultrasound when added to conventional assessment tools. Ann Rheum Dis 2010;69:417-9.
20 Boutry N, do Carmo CC, Flipo RM et al. Early rheumatoid arthritis and its differentiation from other joint abnormalities. Eur J Radiol 2009;71:217-24.
21 Filer A, de Pablo P, Allen G et al. Utility of ultrasound joint counts in the prediction of rheumatoid arthritis in patients with very early synovitis. Ann Rheum Dis 2011;70:500-7.
22 Hülsemann JL, Zeidler H. Diagnostic evaluation of classification criteria for rheumatoid arthritis and reactive arthritis in an early synovitis outpatient clinic. Ann Rheum Dis 1999;58:278-80.
23 Soderlin MK, Borjesson O, Kautiainen H et al. Annual incidence of inflammatory joint diseases in a population based study in southern Sweden. Ann Rheum Dis 2002;61:911-5.
24 Corrao S, Calvo L, Licata G. The new criteria for classification of rheumatoid arthritis: What we need to know for clinical practice. Eur J Intern Med 2011;22:217-9.
25 Chandra PE, Sokolove J, Hipp BG et al. Novel multiplex technology for diagnostic characterization of rheumatoid arthritis. Arthritis Res Ther 2011;13:R102.
26 M?kinen H, Kaarela K, Huhtala H et al. Do the 2010 ACR/EULAR or ACR 1987 classification criteria predict erosive disease in early arthritis? Ann Rheum Dis 2013;72:745-7.
27 Humphreys JH, Verstappen SM, Hyrich KL et al. 2010 ACR/EULAR classification criteria for rheumatoid arthritis predict increased mortality in patients with early arthritis: results from the Norfolk Arthritis Register. Rheumatology (Oxford) 2013 Mar 5. [Epub ahead of print] doi:10.1093/rheumatology/ket113.
Conflict of Interest:
None declared