Article Text
PDF
Abstract
Objectives There is an unmet need for accurate and user-friendly definitions of systemic lupus erythematosus (SLE) disease activity and remission. We aimed to derive and validate the SLE Disease Activity Score (SLE-DAS) definitions for disease activity categories and clinical remission state.
Methods Derivation was conducted at Padova Lupus Clinic (Italy). Validation was prospectively performed at Cochin Lupus Clinic (France) and by post hoc analysis of BLISS-76 trial. At each clinic, an expert classified patients in three categories: remission, mild or moderate/severe activity. The SLE-DAS cut-offs were derived using the receiver operating characteristic curve analysis in Padova cohort; its performance was assessed against expert classification in Cochin cohort and British Isles Lupus Assessment Group (BILAG) index in BLISS-76. Gold standard for clinical remission state was the fulfilment of Definition Of Remission In SLE. A Boolean and an index-based definitions of remission were sustained by chi-square automatic interaction detection algorithm. An SLE-DAS online calculator was developed and tested.
Results We included 1190 patients with SLE: 221 in the derivation cohort and 969 in the validation cohorts (150 from Cochin; 819 from BLISS-76). Derived cut-offs were: remission, SLE-DAS ≤2.08; mild activity, 2.08<SLE-DAS≤7.64; moderate/severe activity, SLE-DAS >7.64. Regarding validation in Cochin cohort, sensitivity and specificity are above 90%, 82% and 95% for remission, mild and moderate/severe activity, respectively. The SLE-DAS Boolean-based and index-based remission showed sensitivity of 100% and specificity above 97%.
Conclusion The SLE-DAS is an accurate and easy-to-use tool for defining SLE clinical remission state and disease activity categories, validated against expert assessment and BILAG.
- systemic lupus erythematosus
- disease activity
- outcomes research
- autoimmune diseases
Data availability statement
Data are available on reasonable request. All data relevant to the study are included in the article or uploaded as online supplemental information. Data are available on reasonable request from AD (ORCID 0000-0003-0548-4983). Reuse of data is not permitted by a third party without authorisation.
Statistics from Altmetric.com
Key messages
What is already known about this subject?
There is an unmet need for accurate and user-friendly definitions of systemic lupus erythematosus (SLE) disease activity categories and remission.
The SLE Disease Activity Score (SLE-DAS) is a composite index with continuous measurement properties that includes important disease activity features absent from SLE Disease Activity Index 2000 (SLEDAI-2K) and presents higher predictive value for damage accrual as compared to SLEDAI-2K.
What does this study add?
Derivation and validation of two SLE-DAS-based definitions of clinical remission presenting very high performance, consistent with Definition of Remission in SLE clinical remission criteria; derivation and validation of SLE-DAS cut-offs for defining categories of active disease presenting a high performance, consistent with expert clinician judgement and British Isles Lupus Assessment Group index; and development of the SLE-DAS online calculator, available at http://sle-das.eu/.
These SLE-DAS definitions enables clinical interpretation of the SLE-DAS scores.
How might this impact on clinical practice or future developments?
The SLE-DAS may facilitate treat-to-target management of patients with SLE, providing a useful instrument for guiding treatment strategies, for outcomes research and identifying candidates for clinical trials.
Introduction
Management of patients with systemic lupus erythematosus (SLE) should aim to achieve remission and to improve long-term patient outcomes.1–7 Additionally, treatment of SLE should be tailored according to the intensity of SLE global disease activity, usually differentiated in mild, moderate and severe.6 8
However, the proposed definitions of remission and SLE disease activity categories are mostly based on the SLE Disease Activity Index 2000 (SLEDAI-2K), which has limitations in accurately defining clinical remission and especially the categories of disease activity.9 10 To compensate the gaps in the SLEDAI-2K, the current definitions of remission include additional items, such as the physician global assessment (PGA), disease features and the ongoing treatment.2 3 11–13 As a result, these definitions are not practical for the use in daily clinical practice. Furthermore, discrimination of disease activity categories based on the SLEDAI-2K score is limited because it dichotomically scores (present/absent) disease activity in each organ domain, not graduating the extent of disease activity within each organ system. In addition, SLEDAI-2K does not include some important lupus features. Categorisation of disease activity applying the British Isles Lupus Assessment Group disease activity index (BILAG) was also proposed, but this is time-consuming and provides organ-based categories and not a global score of disease activity.6 8 14 Furthermore, the BILAG might classify the same level of disease activity in different categories, as this classification is based on the change from the previous month and not strictly on the actual activity at the time of the visit. Hence, there is an important unmet need for more reliable and user-friendly definitions of SLE clinical remission and disease activity categories.
The SLE Disease Activity Score (SLE-DAS) is a validated instrument for measuring global disease activity, with 17 weighted clinical and laboratory parameters, including continuous measures for arthritis, proteinuria, thrombocytopaenia and leucopenia, with the other items scored dichotomously. Differently from SLEDAI, the SLE-DAS gives lower scores to mucocutaneous than systemic vasculitis and to localised skin rash than to generalised skin rash, hence improving the weighting system. Importantly, infrequent but important SLE manifestations absent from SLEDAI are included in SLE-DAS, namely ophthalmological (in neuropsychiatric item), cardiac/pulmonary involvement, gastrointestinal (in systemic vasculitis item) and haemolytic anaemia. Scoring of SLE-DAS with its online calculator is user-friendly. In longitudinal follow-up of patients in two clinical cohorts, we previously assessed the performance of SLE-DAS to discriminate a clinically significant change in SLE disease activity and found in the validation cohort that a score change ≥1.72 had a 95.5% sensitivity and 98.2% specificity for worsening, and an 89.5% sensitivity and 100% specificity for improvement.15–18 As compared to SLEDAI-2K, the SLE-DAS had a higher accuracy in measuring SLE disease activity, a better sensitivity-to-change and a higher predictive value for damage accrual.15
In this study, we aimed to: (1) define SLE clinical remission state based on SLE-DAS; (2) derive and validate the SLE-DAS cut-off values in defining SLE disease activity categories; (3) develop an SLE-DAS online calculator and test its reliability.
Patients and methods
Study populations
In this observational multicentre study, we included consecutive SLE patients fulfilling the American College of Rheumatology (ACR'97) and/or Systemic Lupus International Collaborating Clinics (SLICC'12) classification criteria and followed at the Padova Lupus Clinic (Division of Rheumatology, University of Padova, Italy) or the Cochin Lupus Clinic (Internal Medicine Department, Cochin Hospital, France).3 19 20 Additionally, we analysed data from the phase III, 76-week BLISS-76 (NCT00410384) trial.21 All patients gave informed consent before inclusion.
Patient’s assessments
The assessment was performed at the first visit occurring from 1 March 2018 to 30 June 2018 in Padova and from 1 June 2020 to 30 October 2020 in Cochin Lupus Clinics. At the inclusion visit, the attending clinician evaluated the SLE disease activity (in the preceding 30 days) using PGA (0–3 points, 10 cm Visual Analogue Scale), SLEDAI-2K and SLE-DAS.15 22 The fulfilment of two definitions of clinical remission was also assessed, that is, the Definition Of Remission In SLE (DORIS) and the definition proposed by Zen et al (Doria).2 3 Both definitions allow ongoing prednisone ≤5 mg/day.2 3 At each centre, the senior clinician expert in SLE, blinded to the DAS and remission states and with the knowledge of laboratory results, classified each patient in one of three categories, according to clinical judgement6 8: (1) remission, (2) mild disease activity and (3) moderate/severe disease activity. As a guiding principle, the expert classified patients according to the organ system with the highest level of disease activity.
Data from intention-to-treat population of the BLISS-76 trial at the time of the baseline study visit were analysed in a post hoc study. For inclusion in BLISS-76, patients were required to have active disease (SLEDAI ≥6). We used BILAG data, assessed at the time of the study visits and scored SLE-DAS retrospectively from the study database (as detailed in online supplemental file).
Supplemental material
Data analysis and statistics
For derivation analyses, we used data from the patients enrolled in the Padova Lupus Cohort. We derived the SLE-DAS cut-off values for defining mild and moderate/severe disease activity categories. In addition, we used two alternative approaches to derive definitions of SLE clinical remission state: an index-based and a Boolean-based definition.
Classification performance was evaluated using robust measures against imbalanced data: sensitivity, specificity and Youden index, which are well-known evaluation metrics and also the geometric mean (G-Mean), which considers the product of sensitivity and specificity, giving the balance between classification performance in both groups, and the discriminant power (DP), which evaluates how well the classification rule distinguishes both groups.23–25
All the cut-offs were derived from receiver operating characteristic (ROC) analysis using bootstrap as a way to evaluate the out-of-sample performance of the cut-off estimation method, simulating its variability. One thousand bootstrap samples were considered and for each bootstrap sample the cut-off was determined maximising one of the criteria: the Youden index, the DP or the G-Mean. The R package cutpointr26 was used. The median and 95% bootstrap CIs for the cut-off values are presented. The area under the ROC curve (AUC) was also considered as a measure of discriminatory ability.
Derivation of the SLE-DAS cut-off values for disease activity categories
We applied ROC curves against the expert clinical judgement for defining the SLE-DAS cut-offs for (1) remission, (2) mild and (3) moderate/severe disease activity. In the ROC analysis, the cut-off values were sequentially determined according to the following dichotomisations: remission versus non-remission and remission/mild disease activity vs moderate/severe disease activity.27
Derivation of SLE-DAS definitions for clinical remission state
Following the suggestions proposed by the DORIS project group,2 we tested two definitions of SLE clinical remission based on SLE-DAS:
Index-based definition, applying: (1) the SLE-DAS upper threshold for remission, derived by ROC curve against the expert clinical judgement and (2) prednisone ≤5 mg/day.
Boolean-based definition of remission: (1) all clinical items of SLE-DAS were required to be absent (allowing to be present only the items for anti-double stranded DNA (dsDNA) antibodies and hypocomplementaemia) and (2) prednisone ≤5 mg/day.
For remission state, DORIS definition was used as gold standard. Agreement between the definitions of remission was measured using Cohen’s kappa coefficient. Additionally, the two definitions were substantiated by applying decision trees, using the chi-square automatic interaction detection (CHAID) algorithm. Decision trees are often used in machine learning as classification procedures. The CHAID algorithm generates a decision tree that provides a classification rule to identify homogeneous mutually exclusive subgroups in relation to a criterion variable (dependent variable), and in accordance with the combination of a range of independent variables (predictors). Decision trees were generated using IBM SPSS Statistics, V.26.
External validation of the SLE-DAS definitions for disease activity categories and clinical remission state
Using data from the patients enrolled in the Cochin Lupus Cohort, performance measures were calculated for: (1) the SLE-DAS index-based and Boolean-based definitions of remission, using the DORIS definition as gold standard; (2) the SLE-DAS cut-off values for defining disease activity categories, using the clinical classification from the expert as gold standard. Agreement between the definitions of remission was measured using Cohen’s kappa coefficient.
Post hoc analysis of the BLISS-76 population at study baseline
We performed a post hoc analysis of the BLISS-76 trial, assessing data from the baseline study visit, where patients were expected to have a high level of disease activity. We analysed the performance of the SLE-DAS cut-off value for moderate/severe disease activity (prospectively validated in the Cochin Lupus Cohort) using the BILAG index as gold standard. Patients were considered to be in moderate/severe disease activity when presenting moderate or severe disease in at least one BILAG domain (≥1 B and/or ≥1 A).
IBM SPSS Statistics, V.26 and R software were used, and p<0.05 was considered statistically significant.
Development and reliability assessment of the SLE-DAS online calculator
Our information technology engineering team developed the SLE-DAS online calculator (http://sle-das.eu/) and we tested its reliability by comparing in each patient the SLE-DAS score calculated using a Microsoft Excel spreadsheet with the SLE-DAS formula15 and applying the SLE-DAS calculator.
Results
Characteristics of the patients
Overall 1190 patients were evaluated: 371 in the prospective clinical cohorts (221 and 150 from Padova Lupus Clinic and Cochin Hospital, respectively), and 819 patients from the retrospective analysis of the BLISS-76 population (table 1).21
Characteristics of the patients included (n=1190)
Derivation and validation of the SLE-DAS cut-offs for SLE disease activity categories
In the derivation cohort, the best cut-off values of SLE-DAS to define each SLE disease activity category were: SLE-DAS≤2.08 for remission; 2.08<SLE-DAS≤7.64 for mild disease activity; SLE-DAS>7.64 for moderate/severe disease activity (table 2). AUC values are high (≥0.9) supporting an outstanding discrimination.28 Furthermore, the DP values associated with the cut-off points 2.08 (remission vs non-remission) and 7.64 (remission/mild vs moderate/severe) are, respectively, 4.71 and 3.70, which indicate a good DP.25
Derivation of the SLE-DAS cut-offs to discriminate dichotomous disease activity categories, using ROC analysis
Sensitivity, specificity and G-Mean are above 90% for remission, above 80% for mild and above 95% for moderate/severe disease activity, both in Padova and Cochin cohorts (table 3). Characteristics of the discordant cases are shown in online supplemental tables S2 and S3.
Performance of SLE-DAS cut-offs for disease activity categories, as compared to physician’s classification in the derivation and validation clinical cohorts, and to BILAG scores in the BLISS-76 population
Definition and validation of SLE clinical remission state according to SLE-DAS
SLE-DAS Boolean-based definition of clinical remission
The proportion of patients in clinical remission as defined by the DORIS and the Doria criteria sets was 64.7% and 74.0% in Padova and Cochin cohorts, respectively. Patients fulfilling the SLE-DAS Boolean-based remission (all clinical items of SLE-DAS=0 and prednisone dose ≤5 mg/day) exactly matched those defined by both DORIS and Doria definitions, without discordant cases.
SLE-DAS index-based definition of clinical remission
All patients in remission according to both DORIS and Doria also fulfilled the SLE-DAS cut-off for remission (SLE-DAS ≤2.08). When the condition of prednisone dose ≤5 mg/day was added to the SLE-DAS index-based cut-off for remission, discordance with DORIS was 0.9% in Padova cohort (ie, one patient with leucopenia of 2.9×109/L and one patient with thrombocytopaenia of 89×109/L, without any other active lupus manifestation) and 0.7% in Cochin cohort (ie, one patient with leucopenia of 2.9×109/L, without any other lupus manifestation).
Performance of Doria, SLE-DAS Boolean-based and SLE-DAS index-based definitions of clinical remission are shown in table 4. The agreement between DORIS and Doria criteria, and between DORIS and SLE-DAS Boolean-based definitions were perfect (κ=1, for both) and almost perfect between DORIS and SLE-DAS index-based remission (κ=0.98, p<0.0001) either in Padova or in Cochin cohort. For the derivation cohort, decision trees generated by CHAID are presented in figure 1. Considering as independent variables the prednisone dose, SLEDAI-2K and the SLE-DAS score of clinical items (irrespectively from anti-dsDNA antibodies and hypocomplementaemia), the CHAID classification rule is the same specified in the Boolean-based definition of remission. Using the SLE-DAS score with all its items, the decision tree classification rule also matches the index-based definition of remission. The CHAID algorithm identified SLE-DAS as the variable most significantly associated with remission (DORIS definition); the second one was the prednisone dose ≤5 mg/day. SLEDAI-2K was not incorporated in any of the trees, which means that its importance in predicting remission is superseded by the other two variables.
Decision tree for predicting remission according to DORIS clinical remission: (A) based on SLE-DAS score and prednisone daily dose; (B) based on clinical SLE-DAS (excluding anti-dsDNA antibodies and hypocomplementaemia) and prednisone daily dose—Padova Lupus Cohort. DORIS, definition of remission In SLE; SLE-DAS, systemic lupus erythematosus disease activity score.
Performance of Doria, SLE-DAS index-based and Boolean-based definitions of clinical remission compared to DORIS clinical remission criteria
Evaluation of SLE-DAS cut-offs for SLE disease activity categories in post hoc analysis of the BLISS-76 population
In the BLISS-76 trial, sensitivity, specificity and G-Mean of the SLE-DAS cut-off to identify moderate/severe disease activity defined by the BILAG score was, respectively, 88.6%, 84.1% and 90.8% (table 3).
The ROC curve and the AUC value of SLE-DAS to detect ≥1 BILAG B are presented in figure 2. Performance of SLE-DAS and SELENA-SLEDAI to identify disease activity categories according to BILAG index and characteristics of the discordant cases are shown in online supplemental tables S1 and S4, respectively.
Receiver operating characteristics curve showing the performance of the Systemic Lupus Erythematosus Disease Activity Score (SLE-DAS) to detect ≥1 BILAG B organ domain score in the BLISS-76 population, AUC=0.948 (95% CI 0.923 to 0.973), p<0.0005. AUC, area under the curve; BILAG, British Isles Lupus Assessment Group.
SLE-DAS online calculator
SLE-DAS scoring using the SLE-DAS online calculator took 1–2 min for each patient (depending on the positive items to enter), and there was no discrepancies with the SLE-DAS score calculated using the Microsoft Excel spreadsheet. The http://sle-das.eu/ website layout of the SLE-DAS online calculator is presented in figure 3.
Compact layout of the SLE-DAS online calculator, freely available at http://sle-das.eu/. SLE-DAS, Systemic Lupus Erythematosus-Disease Activity Score.
Discussion
In this study, we derived two SLE-DAS-based definitions of SLE clinical remission, one index-based and another Boolean, both showing in external validation a very high performance, consistent with the DORIS clinical remission criteria. Additionally, we derived the SLE-DAS cut-offs for defining categories of active disease, that showed in the validation cohorts a high performance, consistent with the expert clinician judgement and the BILAG index. These definitions can provide guidance to the clinicians using SLE-DAS in the treat-to-target strategy and adjusting treatment according to the level of disease activity in the management of patients with SLE (figure 4).
Cut-offs of SLE-DAS for (A) disease activity categories and (B) SLE-DAS remission state definitions. SLE-DAS, Systemic Lupus Erythematosus-Disease Activity Score.
The remission state is the preferred target in the management of patients with SLE; however, low disease activity is a good alternative goal if remission cannot be achieved.6 12 Applying the SLE-DAS clinical remission definition, this target was fulfilled by most patients from the clinical cohorts in this study, hence suggesting it is an achievable target in clinical practice. The SLE-DAS definitions of clinical remission do not require the PGA (included in the DORIS definition), that needs standardisation to improve its reliability.29 The SLE-DAS index-based definition was discordant with the DORIS criteria in just three cases (<1% of cases), that only presented mild cytopaenias with normal serology, not requiring specific treatment.6 8 Conversely, the DORIS and Doria clinical remission criteria also allow ongoing mild leucopenia (3.1–3.9×109/L), despite this is counted as lupus manifestations in the SLE classification criteria sets.19 20 30 31 In the SLE-DAS clinical remission definitions, as in the DORIS and Doria criteria, we included the requirement of a prednisone dose ≤5 mg/day. This is important in order to ensure that disease activity is not masked by a high dose of glucocorticoids, as well as because prednisone >5 mg/day is an independent predictor of poor long-term outcome.2 32–34
Accurate definition of active SLE categories is useful as inclusion criteria in clinical trials, in guiding treatment strategies and in defining disease outcomes. Previous attempts to categorise the extent of SLE activity were based either on SLEDAI, BILAG or type of active organ manifestations.6 However, SLEDAI-based categorisation is hampered by the dichotomous nature of its scoring instrument and the weights of the different items which can be inaccurate in defining the actual disease activity. The EULAR recommendations further propose to categorise patients with SLE according to the type and severity of organ involvement, such as classifying localised lupus rashes as mild, while generalised rashes are classified as moderate; thrombocytopaenia is categorised as mild, moderate or severe according to the platelet count.6 There are also some controversial issues with BILAG classification, such as scoring a BILAG-defined severe arthritis variably as ‘B’ or ‘A’ depending on whether the clinician judges it as improving or not. By contrast, SLE-DAS allows a continuous and more objective measure of a core of disease activity manifestations, such as nephritis, arthritis, leucopenia and thrombocytopaenia. Thus, SLE-DAS was expected to allow a more appropriate categorisation of SLE activity. This was confirmed by our study, where the SLE-DAS cut-offs showed an excellent discrimination among the disease activity categories defined according to the expert judgement in a clinical setting or the BILAG index in a clinical trial setting.
Importantly, the online SLE-DAS calculator showed to be reliable and user-friendly. Scoring of SLE-DAS should be feasible in daily clinical practice, as it requires a similar workup time to that of SLEDAI-2K. Notably, applying the SLE-DAS is practical to identify patients in clinical remission state, being consistent with the DORIS and Doria definitions of remission.
Limitations of our study include the low prevalence of patients with high disease activity in the clinical cohorts, hindering the possibility to distinguish between moderate and severe activity categories in these settings. Furthermore, as there is no gold-standard definition for categories of active disease, we used as comparator the expert clinician’s categorisation, which can be subjective. Also the DORIS criteria require better validation. However, we showed that the SLE-DAS categories perform equally well when compared with BILAG, which is a standardised measure. The SLE-DAS was retrospectively scored from the BLISS-76 database, using all the information available from SELENA-SLEDAI, BILAG and laboratory results, inferring the number of swollen joints and skin rashes extension.
Strengths of this study include its large multicentre and multiethnic population from both clinical practice and clinical trial settings, the blinding of the experts’ clinical judgement to the DAS, the comparison against the BILAG for validation of active disease categories and the DORIS criteria for clinical remission state, and the excellent performance in the SLE-DAS definitions of both clinical remission and disease activity categories.
In conclusion, the SLE-DAS is an accurate and user-friendly instrument for classifying SLE in clinical remission state or in different categories of disease activity. The SLE-DAS may facilitate the treat-to-target strategy in the management of patients with SLE, providing a useful instrument in guiding disease treatment, defining new SLE outcomes, and identifying candidates for clinical trials.
Data availability statement
Data are available on reasonable request. All data relevant to the study are included in the article or uploaded as online supplemental information. Data are available on reasonable request from AD (ORCID 0000-0003-0548-4983). Reuse of data is not permitted by a third party without authorisation.
Ethics statements
Patient consent for publication
Ethics approval
This study was approved by the Ethics Committee of the ‘Azienda Ospedaliera-Università degli Studi di Padova’, Padova, Italy. This project adheres to the principles of the Declaration of Helsinki and was approved by the local ethics committee. Informed consent was obtained from all patients before any study procedures.
Acknowledgments
The authors would like to thank to GlaxoSmithKline (Uxbridge, UK) for granting access to the data from the BLISS-76 (NCT00410384) trial through the Clinical Study Data Request consortium.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Josef S Smolen
AD and LSI contributed equally.
Presented at These results were presented, in part, at the Annual European Congress of Rheumatology 2021 and published as a conference abstract (Jesus D, et al. Ann Rheum Dis 2021;80[Suppl1]:182-3).
Contributors DJ, AD and LSI contributed to the conception and design of the project, analysis and interpretation of data, drafting and critical revision of the manuscript; AM and CH contributed to the design of the project, statistical analysis, interpretation of data and critically revised the manuscript; PT, VA and NC developed the SLE-DAS online calculator and contributed to the analysis and interpretation of data and critically reviewed the manuscript; MZ, ML and LI contributed to patients follow-up in the Padova Lupus Clinic, analysis and interpretation of data and critically revised the manuscript; LI scored the SLE disease activity categories in Padova Lupus Cohort; ML, VLG and NC-C contributed to patients follow-up in the Cochin Lupus Clinic and contributed to the analysis and interpretation of data and critically revised the manuscript; NC-C also scored the SLE disease activity categories in the Cochin Lupus Clinic; all the authors approved the final version of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.