Article Text


Extended report
Remission and radiographic outcome in rheumatoid arthritis: application of the 2011 ACR/EULAR remission criteria in an observational cohort
  1. Siri Lillegraven1,2,
  2. Femke HM Prince1,
  3. Nancy A Shadick1,
  4. Vivian P Bykerk1,
  5. Bing Lu1,
  6. Michelle L Frits1,
  7. Christine K Iannaccone1,
  8. Tore K Kvien2,
  9. Espen A Haavardsholm2,
  10. Michael E Weinblatt1,
  11. Daniel H Solomon1
  1. 1Division of Rheumatology, Brigham and Women's Hospital/Harvard Medical School, Boston, Maryland, USA
  2. 2Department of Rheumatology, Diakonhjemmet Hospital, Oslo, Norway
  1. Correspondence to Siri Lillegraven, Division of Rheumatology/Section of Clinical Sciences, Alumnae Hall – Room 305, Brigham and Women's Hospital, 75 Francis Street, Boston, MA 02115, USA; siri.lillegraven{at}


Objectives One goal of remission in rheumatoid arthritis (RA) is to halt joint damage. The authors assessed the progression of radiographic joint damage among RA patients in remission by the new ACR/EULAR criteria (Boolean approach) compared with remission thresholds for the simplified disease activity index (SDAI), clinical disease activity index (CDAI) and disease activity score based on 28 joints and C-reactive protein (DAS28-CRP) in an observational cohort, and evaluated the relationship between time in remission and radiographic joint damage.

Methods 535 RA patients underwent physical examination and laboratory assessment at baseline, 1 and 2 years. Radiographs at baseline and 2 years were scored by the van der Heijde modified Sharp score (TSS). Positive likelihood ratios for a good radiographic outcome (change in TSS <1 unit/year) were calculated for each of the remission criteria. Radiographic progression was compared between patients in remission at none, one, two and three visits by χ2 goodness of fit statistics.

Results 20% of patients in ACR/EULAR remission at baseline had radiographic progression, 24% in SDAI remission, 19% in CDAI remission and 30% of patients in DAS28–CRP remission. The positive likelihood ratio for good radiographic outcome was 2.6 for ACR/EULAR criteria, 2.1 for SDAI, 2.8 for CDAI and.1.5 for DAS28–CRP. Reduced radiographic progression was observed for patients with an increasing number of visits in remission (p<0.003 for all criteria, χ2 goodness of fit statistics).

Conclusions Patients with RA in remission by any established criteria can experience radiographic progression. An increased number of visits in remission was associated with reduced radiographic damage.

Statistics from

Rheumatoid arthritis (RA), a chronic disease affecting both joints and extra-articular organs, can cause substantial joint damage and loss of function.1 With more effective drugs and treatment strategies, remission has become the treatment target.2 3

Several remission criteria are already established in RA. The disease activity score in 28 joints (DAS28),4 5 the clinical disease activity index (CDAI)6 and the simplified disease activity index (SDAI)7 are disease activity indices with cut points defining remission. The 1981 American College of Rheumatology (ACR) remission criteria8 are the only RA remission criteria incorporating duration of remission, but these criteria are not commonly used. Studies showing that some patients assessed to be in remission by current criteria have progressive joint damage9,,11 have motivated the Outcome Measures in Rheumatology (OMERACT) network, the ACR and the European League Against Rheumatism (EULAR) to develop a new definition of remission.12

The ACR/EULAR provisional definition of remission in RA was developed using data from four clinical trials.12 To be classified as in remission according to these criteria, patients must have swollen joint count (SJC), tender joint count (TJC), C-reactive protein (CRP) (mg/dl) and patient global assessment on a 10 cm visual analogue scale of one or less.12 Results from the development of the new criteria show a likelihood ratio of 2.9 for identification of patients with good radiographic outcome after 1 year, and a likelihood ratio of 7.2 for both good radiographic and good functional outcome.12 These findings have not yet been validated in other datasets, and it has been especially noted that the new remission criteria should be validated in observational cohorts, as the patients participating in clinical trials are not representative of the whole RA population.13 14

In the development of the new criteria, duration of remission was excluded at an early stage based on advice from the expert committee,12 and no testing of the added value of incorporating duration into the criteria was performed. Earlier studies of previous RA remission criteria suggest that duration of remission is associated with radiographic outcome,10 15 16 and one of the questions that has been raised after the publication of the new criteria is whether time spent in remission would improve the performance of the criteria.13 14

The main objective of this study was to examine the performance of the new ACR/EULAR criteria in an observational cohort in comparison with other established RA remission criteria, focusing on their relationship with radiographic progression. We also studied the incremental predictive value of including time in remission in the assessment of RA patients.

Material and methods

Patients and examinations

Patients were selected from the Brigham Rheumatoid Arthritis Sequential Study (BRASS), a single-centre observational cohort of over 1100 RA patients.17 All patients included in the analyses were recruited between 2003 and 2006, diagnosed by board-certified rheumatologists as having RA and the patients fulfilled the 1987 ACR classification criteria for RA.18 The patients were assessed by biochemical markers (annually), clinical examinations with SJC and TJC (annually) and patient reported outcome measures (semi-annually). Treatment was given according to the clinical practice of the rheumatologist, with visits between the annual data collection in BRASS as found necessary by the physician. There was no predefined structured treatment strategy. Conventional radiographs of hands and wrists were performed at baseline and the 2-year follow-up. Only patients with radiographic follow-up were included in these analyses (n=535). Patients without radiographs had either declined to participate in radiographic examinations or the radiographs were not of sufficient quality to be scored. The study was approved by the Institutional Review Board of Brigham and Women's Hospital and all patients gave written informed consent.


The radiographs were scored according to the van der Heijde modified Sharp scoring method by four trained radiologists blinded to the sequence of the radiographs.19 The choice of more than one reader was for practical reasons. The intraclass correlation coefficient for the van der Heijde modified Sharp score (TSS) interreader reliability was 0.93 for baseline and 0.85 for change scores (average measures intraclass correlation coefficient, 40 patients scored by two of the readers). Change in TSS during the 2 years of follow-up was the main outcome of the study, and was divided by years of follow-up for calculation of the annual rate of change. Patients with 1 or more unit change in TSS per year were classified as ‘progressors’, based on the suggestion of 1 unit change as the lowest value of minor radiographic change.20 Secondary analyses were performed with the smallest detectable change as a cut-off, calculated with the standard deviation (SD) of the differences between change scores of two reading sessions (as described by Bruynesteyn et al).21 The median (inter-quartile range, IQR) time between clinical visit at baseline and radiographs was 0 (0, 0.1) years, but a few patients had radiographs taken closer to the second clinical visit. We therefore assessed the relationship between remission status and radiographic progression in two sensitivity analyses: excluding patients with radiographs taken more than 3 months from clinical baseline and adjusting clinical baseline in 22 patients with another visit closer to the radiographic baseline.

Remission criteria and sustained remission

Four different remission criteria were evaluated in this study. DAS28–CRP4 5 22 was calculated with CRP (mg/l), TJC and SJC (based on assessment of 28 joints) and the patients' assessment of the disease activity on a visual analogue scale (VAS on a 0–100 mm scale). Remission was defined as a disease state with a DAS28–CRP value of less than 2.6. SDAI was calculated with TJC and SJC (28 joints), patient and physician global assessment of disease (VAS on a 0–10 cm scale) and CRP (mg/dl), and remission was defined as a disease state with SDAI value of 3.3 or less.7 CDAI includes similar variables to SDAI, but without CRP to facilitate use in clinical practice. The cut-off for CDAI remission is 2.8 or less.6 7 The ACR/EULAR remission criteria require the patients to have one or fewer tender joints, one or fewer swollen joints, CRP of 1 or less (mg/dl) and patient global assessment of 1 or less (on a 0–10 cm VAS scale). The ACR/EULAR remission workgroup also endorses SDAI remission for clinical trials and CDAI remission for clinical practice.12

Statistical analyses

All analyses were performed using SPSS for Windows version 15. Baseline characteristics for all patients and patients fulfilling the different remission criteria were presented by median (IQR) or absolute frequencies as appropriate. The median (IQR) subsequent change in TSS was calculated for each remission group at baseline. Annual change in TSS for each patient fulfilling DAS28–CRP, SDAI, CDAI and ACR/EULAR remission at baseline was illustrated in a cumulative probability plot.

To compare the performance of the different remission criteria with each other and the results in the recent publication of the ACR/EULAR criteria,12 we calculated performance characteristics for the remission criteria. Sensitivity, specificity and likelihood ratios were calculated for the ability of each remission criteria to identify the patients with good outcome, defined as absence of radiographic progression (<1 unit change in TSS/year). The positive likelihood ratio is calculated as sensitivity divided by (1–specificity). The association between remission and lack of progression was also assessed as the odds ratio (OR) by univariate logistic regression models with absence of radiographic progression (defined as <1 unit change in TSS/year) as the dependent variable and remission status at baseline as the independent variable. To relate the performance of the remission criteria to the absence of inflammation on clinical examination, we calculated the same values (ie, sensitivity, specificity, positive likelihood ratio, OR) for absence of swollen joints (SJC 0) at baseline.

Our second objective was to study how duration of remission affects radiographic outcome in patients in remission according to different criteria. Most patients had three visits—baseline, 1 and 2 years. As a model of time spent in remission, patients were grouped according to the number of visits in remission during the 2-year follow-up period (none to three), and the percentage of patients classified as progressors was calculated for each group. The overall trend for the effect of time in remission in different criteria was assessed by χ2 goodness of fit statistics. Positive likelihood ratios, sensitivity, specificity and unadjusted logistic regression models for good radiographic outcome (annual change in TSS <1) were then calculated for each remission criterion, requiring the patients to have more than one visit (of the three possible) in remission during radiographic follow-up. The two visits did not have to be consecutive, as the dataset did not include enough patients with two consecutive visits in remission.


The median (IQR) age for the 535 patients was 59 years (50–67 years), median disease duration was 11 years (IQR 4–23 years), 442 (83%) were women, 359 (67%) were anticyclic citrullinated peptide antibody positive and 339 (63%) were rheumatoid factor positive. Full patient characteristics for all patients and patients in remission by different criteria at baseline are presented in table 1. None of the patients was in remission by SDAI, CDAI or ACR/EULAR criteria without also fulfilling the DAS28–CRP criteria. The overlap between the remission criteria is detailed in supplementary table 1, available online only. The median follow-up time between radiographs was 2.1 years (IQR 1.9–2.4 years). The smallest detectable change for TSS was 6.2 units.

Table 1

Baseline demographic and treatment data for all patients, and for patients assessed to be in remission according to DAS28–CRP, SDAI, CDAI and ACR/EULAR at baseline

Radiographic progression in the different remission criteria

The median (IQR) annual change in TSS for patients in remission at baseline was 0.0 (0.0–1.2) for DAS28–CRP, 0.0 (0.0–1.0) for SDAI, 0.0 (0.0–0.4) for CDAI and 0.0 (0.0–0.5) for ACR/EULAR criteria and the mean (SD) annual change in TSS was 0.93 (2.9) for DAS28-CRP, 0.65 (2.4) for SDAI, 0.37 (1.5) for CDAI and 1.08 (3.9) for ACR/EULAR criteria. A cumulative probability plot showing the individual data for all patients assessed to be in remission by DAS28–CRP, SDAI, CDAI and ACR/EULAR at baseline is presented in figure 1. Numerically, more patients in DAS28–CRP remission were classified as progressors for TSS than by the other remission criteria, both in the primary analysis and the secondary analysis (table 2). Sensitivity analyses, as described in the Methods section, were performed to assess whether a delay in radiographs affected the outcome, with similar results for all approaches (see supplementary table 2, available online only).

Figure 1

Cumulative probability plot for patients in remission according to DAS28–CRP, SDAI, CDAI and ACR/EULAR criteria at baseline. Each dot represents the annual change in TSS (y-axis) for an individual patient. ACR, American College of Rheumatology; CDAI, clinical disease activity index; DAS28–CRP, disease activity score based on 28 joint count and C-reactive protein; EULAR, European League Against Rheumatism; SDAI, simplified disease activity index; TSS, van der Heijde modified Sharp score.

Table 2

Number of patients classified as progressors according to remission status (DAS28–CRP, SDAI, CDAI and ACR/EULAR) at baseline

Positive likelihood ratios for non-progression of TSS (good radiographic outcome, table 3) were highest for CDAI (2.8) and ACR/EULAR remission criteria (2.6), while DAS28–CRP had the lowest positive likelihood ratio (1.5). CDAI also had the highest OR for the prediction of good radiographic outcome in unadjusted logistic regression analyses (table 3).

Table 3

The performance of remission criteria at baseline for the identification of patients without subsequent radiographic progression

We assessed the radiographic change in patients without clinical inflammation at baseline (SJC 0). Of 93 identified patients, 21 (23%) had 1 or more unit change in TSS per year, and seven (8%) were classified as progressors according to the smallest detectable change. The positive likelihood ratio for the detection of patients with no progression (good radiographic outcome) was 2.3 (table 3).

Time in remission and radiographic outcome

The effect of time in remission on radiographic outcome was assessed by grouping patients according to time spent in remission (number of visits in remission) during follow-up. When we calculated the percentage of patients classified as progressors for each group, CDAI at three visits (n=7) was the only group without radiographic progression (figure 2). Patients had significantly less radiographic progression with more visits in remission for all remission criteria (figure 2, and see supplementary figure, available online only). When assessing patients with more than one visit in remission during follow-up (visits did not have to include baseline or be consecutive), both positive likelihood ratios and OR were numerically higher (table 4) than when only assessing baseline remission.

Figure 2

Non-progressor and progressor (progression defined as annual change in TSS ≥1 unit) groups according to visits in remission. p Values for trend from χ2 goodness of fit tests. (A) DAS28–CRP, p=0.003. (B) SDAI, p<0.001 (five or fewer patients had three visits in remission). (C) CDAI, p<0.001. (D) ACR/EULAR, p<0.001 (five or fewer patients had three visits in remission). ACR, American College of Rheumatology; CDAI, clinical disease activity index; DAS28–CRP, disease activity score based on 28 joint count and C-reactive protein; EULAR, European League Against Rheumatism; SDAI, simplified disease activity index; TSS, van der Heijde modified Sharp score.

Table 4

The performance of remission criteria when patients with at least two visits in remission during the radiographic follow-up period are selected (patients did not have to be in remission at baseline, visits did not have to be consecutive)


With increasing treatment options, remission has become the goal of contemporary RA treatment. In this study, we examined the performance of remission criteria in an observational cohort of patients with RA, focusing on the new ACR/EULAR remission criteria and progression of radiographic joint damage. The performance of the ACR/EULAR criteria with a Boolean approach12 was comparable to the results from the clinical trials database where it was developed. In addition, our data support that duration of remission plays a role with regard to radiographic progression and adds information to all remission criteria.

The ACR/EULAR committee developing the new classification criteria had access to data from several large clinical trials during their work. Three variables (SJC, TJC and CRP) were predefined to be included in the criteria. The dataset was then used to assess the added value of other core set variables (pain, patient and physician global) in the prediction of good radiographic and functional outcome. Our study had approximately the same number of patients in remission, and twice as long follow-up time for radiographs. The positive likelihood ratio for the ACR/EULAR criteria in the observational BRASS data was comparable to that presented from trial data (2.7 vs 2.9). None of the remission criteria were able to exclude all patients with future radiographic progression. Although not a remission criterion, the absence of swollen joints had a comparable performance with several of the remission criteria with a positive likelihood ratio of 2.3 for the detection of good radiographic outcome, supporting that the absence of clinical synovitis is important to prevent joint damage.

The results also, as did the publication of the ACR/EULAR criteria,12 illustrate that a significant proportion of patients who do not fulfil the remission criteria still experience a good outcome of their disease, especially for radiographic joint damage, but also when combining radiographic damage and functional status. A by-product of more stringent remission criteria as a treatment target will be more patients in low disease activity with good disease outcome, potentially risking overtreatment. It would be of interest to identify predictors of good disease outcome in such patients.

Sustainability has not been a part of any of the RA remission definitions developed since the 1981 ACR remission criteria.8 The consensus among experts has been that this should not be included in current remission criteria for clinical trials,12 even if the sustainability of an achieved state has been considered important when reporting results from clinical trials,23 and studies have shown that approximately 10% of patients can achieve sustained drug-free remission.24 Our study shows that time in remission also adds important information to the new criteria, with significantly less radiographic progression with more time in remission, and a trend towards higher positive likelihood ratios in patients with at least two visits in remission during follow-up. Adding time in remission as a feature of the remission criteria would probably improve the ability to identify patients with good future outcome as measured by no radiographic progression.

This study had certain strengths and limitations. BRASS has a relatively high number of patients with 2-year radiographic follow-up. The patients in this analysis represent a broad spectrum of disease activity and disease duration, compared with many clinical trials that are performed in a limited patient population with high disease activity and short disease duration. Our findings in this setting strengthen the applicability of the ACR/EULAR criteria in clinical practice, but the findings might have been different in a patient population of early RA patients, in whom the likelihood of remission and its influence on radiographic progression might have been different. Time in remission could have been modelled more precisely if more frequent examinations had been performed. The analyses presented in this paper are crude models of persistent remission, probably underestimating the effect of true sustained remission. All four readers who scored the radiographs were trained radiologists, and the interreader reliability was very good. However, having one reader only or all radiographs scored twice by two different readers would have been more optimal, and it is likely that we lost some sensitivity to change by increased variance in the scorings. Some additional patients might have been classified as progressors if feet radiographs had been available. This limitation contributed to a conservative estimate of progressors in each group. Finally, a larger number of patients in each remission group would have increased the ability to observe differences between the remission criteria.

In this observational cohort, the new ACR/EULAR criteria for remission performed similarly to the trial data from which they were developed. None of the tested remission criteria were able to exclude all patients with significant radiographic damage over 2 years. These findings support studies concluding that the absence of inflammation by modern imaging assessments, such as MRI and ultrasonography, might be considered as a component of remission in the future, especially with better and more feasible imaging techniques.9 25 26 More studies are needed to explore whether imaging-guided disease activity assessment will prove beneficial in everyday clinical practice. The results also support that time is an important aspect of remission.


View Abstract

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

    • Web Only Data - This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.


  • Competing interests None.

  • Ethics approval This study was conducted with the approval of the Institutional Review Board, Brigham and Women's Hospital.

  • Patient consent Obtained.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.