Article Text
PDF
Abstract
Objective To determine cutoff values for defining the state of high disease activity (HDA) in juvenile idiopathic arthritis (JIA) using the Juvenile Arthritis Disease Activity Score (JADAS).
Methods For the selection of cutoff values, data from a clinical database including 609 patients were used. Optimal cutoff values were determined against external criteria by calculating the 25th and 10th centile of cumulative score distribution and through receiver operating characteristic curve analysis. External criteria were based on the therapeutic decision made by the attending doctor. Cross-validation was performed using five patient samples that included 1421 patients.
Results The optimal cutoff values were those obtained through the 90% fixed sensitivity method. The selected JADAS cutoff values were the following: 4.2 and 8.5 for JADAS27 in oligoarthritis and polyarthritis, respectively; 4.2 and 10.5 for both JADAS10 and JADAS71 in oligoarthritis and polyarthritis, respectively. In cross-validation analyses, the cutoff values showed strong ability to discriminate between different levels of American College of Rheumatology paediatric response in two clinical trials and could predict worse functional and radiographic outcome.
Conclusions Cutoff values for classifying HDA in JIA using the JADAS were developed. In cross-validation analyses, they proved to have good construct and discriminant validity and ability to predict disease outcome.
Statistics from Altmetric.com
Introduction
Assessment of disease activity is a fundamental component of the clinical evaluation of children with juvenile idiopathic arthritis (JIA) because persistently active disease plays a major role in causing joint damage and physical functional disability.1 Recently, the first composite disease activity score for JIA, the Juvenile Arthritis Disease Activity Score (JADAS), was reported.2 To facilitate interpretation of JADAS scores, criteria (ie, cutoff values) are needed for identifying high and low levels of JIA activity.
The purpose of this study was to determine and validate cutoff values in the JADAS that correspond to the state of high disease activity (HDA).
Patients and methods
JADAS calculation
The JADAS is calculated as the arithmetic sum of the scores of the following variables: physician global rating of disease activity; parent/child rating of well-being; active joint count, assessed in 71, 27, or 10 joints (JADAS71, JADAS27, and JADAS10, respectively); and erythrocyte sedimentation rate, normalised to a 0–10 scale.2
Study population used for selection of cutoff values
For the selection of cutoff values, we used data from 609 children with JIA3 who had undergone 1814 visits between 2007 and 2009.4 ,5 Children with systemic arthritis (n=46), rheumatoid factor-positive (n=9) and -negative (n=138) polyarthritis or extended oligoarthritis (n=96) were included in the polyarthritis group. The oligoarthritis group included children with persistent oligoarthritis (n=267). Children classified in other JIA categories (n=53) were assigned to the polyarthritis or oligoarthritis group based on the number of affected joints (≤4 or >4, respectively). Children with systemic JIA and active systemic manifestations were excluded. The study protocol was approved by the Istituto G Gaslini institutional review board.
Definition of HDA state
All patients’ visits were examined to identify those patients who met the criteria for HDA. To ensure face validity of the criteria, it was decided that they had to be based on the therapeutic decision made by the attending paediatric rheumatologist. The criteria for HDA in oligoarthritis were (a) intra-articular corticosteroid (IAC) administration in ≥1 joint; (b) start of a disease-modifying antirheumatic drug or (c) start of a biological agent. The criteria for HDA in polyarthritis were: (a) IAC administration in ≥3 joints; (b) start of a disease-modifying antirheumatic drug; (c) start of a biological agent or (d) start of systemic corticosteroid therapy. All visits which did not meet the criteria for HDA were considered as reflecting low/moderate disease activity.
Study populations used for cross-validation of cutoff values
Five patient samples (not used in calculation of cutoff values) were used to cross-validate the cutoff values. The first and second samples comprised patients enrolled in controlled trials of methotrexate (MTX) (n=595)6 and abatacept (n=190).7 The third sample included 175 patients seen between 2009 and 2012 and followed up for at least 6 months.8 The fourth sample included 358 patients who underwent two or more visits between 1997 and 2002.9 The fifth sample consisted of 60 of 103 patients included in a study on the paediatric adaptation of the Sharp/van der Heijde score.10
Statistical analyses used for cutoff selection
Optimal cutoff values were determined against external criteria (ie, the states of HDA, defined as above) by calculating the 25th and 10th centiles of score distribution and through receiver operating characteristic (ROC) analysis. In ROC analysis, three methods were applied: (1) the closest point to (0,1)—that is, the point where the shoulder of the ROC curve is closest to the left upper corner of the graphic; (2) the fixed 90% specificity and (3) the fixed 90% sensitivity. Only one observation for each person was used in selection and validation of cutoff values.
Cross-validation study
Cross-validation of cutoff values was based on assessment of construct, discriminant and predictive validity through the following approaches. (1) Calculation of the percentage of patients who had a JADAS above the HDA cutoff values in clinical trials in relation to level of improvement according to the American College of Rheumatology paediatric criteria11 in the MTX6 and abatacept7 trials. (2) Calculation of the percentage of patients who had a JADAS above the HDA cutoff values in relation to subjective assessment of disease states5 ,12 by doctors and parents. (3) Assessment of the ability of cutoff values to predict lack of attainment of inactive disease13 and achievement of worse functional status on the Childhood Health Assessment Questionnaire (CHAQ).14 ,15 (4) Assessment of the ability of cutoff values to predict progression of radiographic joint damage.10
Quantitative and percentage data were compared by Mann–Whitney U test and χ2 test, respectively. The statistical packages used were Statistica (StatSoft) and Stata, V.7 (StataCorp).
Results
JADAS value by treatment decision
The JADAS10 values by therapeutic intervention are presented in online supplementary table S1.
Selection of the optimal cutoff values for classification of HDA
The cutoff values obtained through the various statistical approaches are presented in table 1. In selecting the final cutoff values, we reasoned that since the ‘gold standard’ was HDA, in order to reduce the risk of misclassifying patients whose disease was active, more importance should be given to sensitivity—that is, to the proportion of patients with active disease who are correctly classified. However, a minimum specificity of 75% was required to minimise the rate of misclassification of patients with low/moderate disease activity as having HDA. Based on these considerations, it was felt that the optimal cutoff values were those provided by the 90% fixed sensitivity method. Notably, all area under the curves were >0.90, which reflects, according to a rule of thumb, ‘outstanding discrimination’.16 The graphs of ROC curves are illustrated in online supplementary figure S1.
JADAS cutoff values for classification of patients into high disease activity state according to five different methods for determining optimal cut-off points
Results of cross-validation analyses
In the MTX and abatacept trial samples, the proportion of patients with JADAS above the HDA cutoff value was greatest among non-responders and lowest among those with 70% improvement (figure 1 and online supplementary figure S2). In the third sample, the percentage of patients with JADAS above the cutoff value was greater among patients judged as having active disease and flare or as not being in an acceptable symptom state, and lower among patients judged as being in remission or in an acceptable symptom state (figure 2 and online supplementary figure S3). In the fourth sample, the percentage of patients with inactive disease or with a CHAQ score of 0 at the final visit was lower among patients who had a JADAS above the cutoff value at the first visit than among those who did not (results not shown). In the fifth sample, the adapted Sharp/van der Heijde score at 3 years was higher among patients who had JADAS above the cutoff values at least twice during the 3 years of observation than in those who did not (online supplementary figure S4).
Percentage of patients with a Juvenile Arthritis Disease Activity Score 10 (JADAS10) above the cutoff values for high disease activity state according to response at 6 months in the methotrexate trial (n=595) as assessed by American College of Rheumatology (ACR) paediatric criteria (not improved, or improved by 30%, 50%, or 70% (ACR 30, ACR 50 and ACR 70, respectively)). p<0.0001 for comparison of response groups.
Percentage of patients who had a Juvenile Arthritis Disease Activity Score 10 (JADAS10) above the cutoff value for high disease activity (HAD) state among patients (n=175) who were judged subjectively as being in the states of remission, persistent activity or disease flare by the doctor (A) or the parent (B). p<0.0001 for comparison of disease states. Remission was defined as ‘Complete absence of symptoms’. Persistent activity was defined as ‘Continuing presence of symptoms’. Disease flare was defined as ‘Recurrence of symptoms after a period of complete well-being’ (see also Ravelli et al10).
Discussion
In this study, we sought to determine the cutoff values on the JADAS that corresponded with the state of HDA. The cutoff values were developed using a routine care population of 609 patients and were cross-validated in five datasets, comprising a total of 1421 patients.
Treatment decisions were used as criteria for HDA. We reasoned that any index of disease activity should reflect clinical practice, and therefore, we focused on real decisions in patient management. We defined as high the level of disease activity demonstrated at a clinic visit where the doctor decided to initiate or perform (in the case of IAC injection) a disease-specific treatment, and as low or moderate the level at a visit where the doctor did not make any major therapeutic interventions. The HDA cutoff values then resulted from discriminant functions that optimally distinguished between these two states.
In cross-validation analyses, the selected cutoff values showed strong ability to discriminate between different levels of American College of Rheumatology paediatric response in two clinical trials. Further evidence of discriminant validity came from the observation that the cutoff values were met more frequently in patients judged by the doctor or the parent as being in the states of persistently active disease and flare or deemed by the parent as not being in an acceptable state than in patients judged as being in the state of remission or in an acceptable state.
Importantly, the cutoff values revealed the ability to predict a worse disease prognosis because patients who had a level of disease activity above their thresholds were less likely to have inactive disease or a CHAQ score of 0 at the last follow-up visit or showed a greater progression of radiographic joint damage.
Some caveats should be considered. We recognise that the aggregation of patients in ad hoc categories based on the number of affected joints is arbitrary. We used an observation-based statistical approach, in which HDA was inferred from a proxy variable (eg, clinician's decisions to start drug treatment). A consensus definition or a judgemental approach (ie, explicitly asking doctors their opinion on what they would consider HDA) might have led to cutoff values with higher face validity and practical relevance.
In summary, we have defined JADAS cutoff levels for classification of HDA in JIA. The cutoff values may support the decision to start a particular treatment for an individual patient and assist in monitoring the disease course over time.
Acknowledgments
The authors thank Drs Marleen Nys and Alan Block, from Bristol Myers & Squibb, for providing access to the data of the abatacept trial.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Files in this Data Supplement:
- Data supplement 1 - Online supplement
Footnotes
-
Handling editor Tore K Kvien
-
Contributors We confirm that all listed authors have provided a significant contribution in the study by participating in design and conduct, data entering, data analysis, manuscript preparation or patient enrolment and assessment.
-
Competing interests None.
-
Ethics approval Istituto Gaslini Ethics Committee.
-
Provenance and peer review Not commissioned; externally peer reviewed.