Article Text
PDF
Abstract
Objectives To derive and validate a new disease activity measure for systemic lupus erythematosus (SLE), the SLE Disease Activity Score (SLE-DAS), with improved sensitivity to change as compared with SLE Disease Activity Index (SLEDAI), while maintaining high specificity and easiness of use.
Methods We studied 520 patients with SLE from two tertiary care centres (derivation and validation cohorts). At each visit, disease activity was scored using the Physician Global Assessment (PGA) and SLEDAI 2000 (SLEDAI-2K). To construct the SLE-DAS, we applied multivariate linear regression analysis in the derivation cohort, with PGA as dependent variable. The formula was validated in a different cohort through the study of: (1) correlations between SLE-DAS, PGA and SLEDAI-2K; (2) performance of SLEDAI-2K and SLE-DAS in identifying a clinically meaningful change in disease activity (ΔPGA≥0.3); and (3) accuracy of SLEDAI-2K and SLE-DAS time-adjusted means in predicting damage accrual.
Results The final SLE-DAS instrument included 17 items. SLE-DAS was highly correlated with PGA (r=0.875, p<0.0005) and SLEDAI-2K (r=0.943, p<0.0005) in the validation cohort. The optimal discriminative ΔSLE-DAS cut-off to detect a clinically meaningful change was 1.72. In the validation cohort, SLE-DAS showed a higher sensitivity than SLEDAI-2K (change ≥4) to detect a clinically meaningful improvement (89.5% vs 47.4%, p=0.008) or worsening (95.5% vs 59.1%, p=0.008), while maintaining similar specificities. SLE-DAS performed better in predicting damage accrual than SLEDAI-2K.
Conclusion SLE-DAS has a good construct validity and has better performance than SLEDAI-2K in identifying clinically significant changes in disease activity and in predicting damage accrual.
- systemic lupus erythematosus
- SLE-DAS
- SLEDAI
- disease activity
- outcomes research
Statistics from Altmetric.com
Key messages
What is already known about this subject?
The SLE Disease Activity Index (SLEDAI) is the most widely used SLE disease activity measure and the core determinant in the SLE Responder Index (SRI) applied as primary endpoint in clinical trials.
The performance of SLEDAI in detecting clinically meaningful changes in disease activity is limited.
What does this study add?
This study presents the derivation and validation of the SLE Disease Activity Score (SLE-DAS), a new continuous global score to assess disease activity in SLE. The SLE-DAS provides a more accurate identification of clinically meaningful changes over time, with a much higher sensitivity as compared with SLEDAI-2K and similar specificity. The SLE-DAS presented higher performance in predicting damage accrual, as compared to SLEDAI-2K.
How might this impact on clinical practice or future developements?
The use of the SLE-DAS in clinical practice and as an outcome in clinical trials will allow a much higher discriminative power to detect clinically meaningful changes in SLE disease activity. Importantly for its use, collection of SLE-DAS items requires a clinical workup time similar to SLEDAI, and much lower than the British Isles Lupus Assessment Group and SRI.
Introduction
The accurate assessment of disease activity in patients with systemic lupus erythematosus (SLE) is a critical tool for clinical practice, observational studies and clinical trials.1 2 Over the last three decades, multiple disease activity indices have been developed. Yet, all these instruments have important limitations, hindering their usefulness.3 Hence, the standardised measurement of SLE disease activity remains a challenging task.4 5
The SLE Disease Activity Index (SLEDAI), which includes the SLEDAI 2000 (SLEDAI-2K) and Safety of Estrogens in Lupus Erythematosus National Assessment (SELENA)-SLEDAI versions, is the most widely used SLE disease activity measure.6–8 Furthermore, SLEDAI is the core determinant in the SLE Responder Index 4 used as primary endpoint in recent clinical trials.9–13 However, the performance of SLEDAI in detecting clinically meaningful changes in disease activity is limited, since each item of SLEDAI is scored dichotomically, with the same numerical weight regardless of the severity of change observed. Only remission of any lupus feature, but not partial improvement, is captured as a change in SLEDAI score; likewise, an increased severity of any previously active manifestation cannot be scored. In addition, potentially severe lupus manifestations, such as haemolytic anaemia, pneumonitis or gastrointestinal involvement, are not scored in SLEDAI.
These limitations have important implications on target-driven management of SLE in daily clinical practice, and in assessing the efficacy of new medications in clinical trials. In fact, despite many promising lupus treatments reaching early-phase human studies during last decades, none except belimumab has yet demonstrated efficacy in phase III trials using these outcome measures, in contrast with the clinician’s impression of efficacy.14–19 Thus, there is an unmet need for tools able to accurately capture clinically meaningful changes in SLE disease activity.
Herein, we describe the development and validation of a new instrument for the measurement of SLE disease activity: the SLE Disease Activity Score (SLE-DAS).20 The aim of SLE-DAS is to provide an accurate, continuous and user-friendly global measure of SLE disease activity with improved sensitivity to change and high specificity, in comparison to SLEDAI-2K.
Patients and methods
Study and settings
Longitudinal cohort study conducted at CHUC Lupus Clinic at Rheumatology Department, Centro Hospitalar e Universitário de Coimbra (CHUC), Portugal (derivation cohort) and at Rheumatology Unit, University of Padova, Italy (validation cohort). This project adheres to the principles of the Declaration of Helsinki and was approved by the local ethics committees. Informed consent was obtained from all patients.
Patients
Consecutive patients with SLE fulfilling American College of Rheumatology (ACR’97) and/or Systemic Lupus International Collaborating Clinics (SLICC’12) classification criteria, enrolled in the CHUC Lupus Cohort from January 2014 to December 2017 and in the Padova Lupus Cohort, from January 2013 to June 2018, were studied.21 22 Data from the CHUC Lupus Cohort were used for derivation and internal validation of the SLE-DAS and data from the Padova Lupus Cohort were used for external validation.
Parameters assessed
Clinical and laboratory assessment considered all manifestations included in SLEDAI-2K and in the current definitions of low disease activity (LDA) or remission, in a total of 17 variables (table 1).8 12 13 Disease activity in the previous 30 days was scored, by one senior rheumatologist with large experience in SLE management in each cohort, using Physician Global Assessment (PGA) (0–3) and SLEDAI-2K (0–105).8 Organ damage was evaluated at each outpatient visit using the SLICC/ACR Damage Index for SLE (SDI).23
Clinical and laboratory parameters attributable to systemic lupus erythematosus (SLE), assessed at each outpatient visit
Moreover, in the Padova Lupus Cohort, the disease activity data at the last follow-up visit were compared with the previous one and the change was classified by the senior physician as reflecting: (1) clinically meaningful improvement; (2) no clinically meaningful change; and (3) clinically meaningful worsening.
Data analysis and statistics
On clinical grounds, we adopted the guiding principle that all variables listed in table 1 ought to be included in the final instrument, given that their relevance is well established in practice.
Derivation of the disease activity score
For derivation of the SLE-DAS, data from the CHUC Lupus Cohort were used. Data from the visit with the highest disease activity during follow-up from each patient were selected for analysis.
Multivariate linear regression was applied, with PGA as dependent variable. All disease manifestations included in table 1 were considered as potential independent variables for the model. Because the precision of an estimated coefficient may be affected if the corresponding variable has little sample variation, and the regression model ability to predict future observations can be compromised increasing the number of variables, only variables with more than 10 observations were included in the multivariate linear regression model. The remaining manifestations were attributed a discretionary weight score by the senior investigator (LSI), taking into account the coefficients of the other variables in the regression model and also the relative impact of these variables in the expert physician’s disease activity assessment.
Several different functions of disease manifestations were sequentially tested in looking for the model with the best possible adjustment to PGA. Urinary protein excretion, platelet and white cell counts were modelled considering their non-linear impact on disease activity assessment as reflected in PGA. Regarding arthritis, two variables were included in the model: a dummy variable indicating the presence or absence of arthritis, and the number of swollen joints, in order to improve the assessment of severity.
Internal validation
The construct validity was assessed by comparing the global score of the SLE-DAS with PGA and SLEDAI-2K, using Spearman’s correlation coefficients , at the last follow-up visit. During follow-up, the ability of SLE-DAS to discriminate a clinically meaningful worsening or improvement in SLE disease activity (defined as a change in PGA≥0.3) was assessed using receiver operating characteristic (ROC) analysis and compared with the performance of SLEDAI-2K. The areas under the ROC curves (AUC) were used to evaluate and compare the discriminative ability of SLE-DAS and SLEDAI-2K. By analyses of ROC curves, ideal cut-off values of ΔSLE-DAS to detect clinically meaningful worsening/improvement were identified, and its sensitivity, specificity, and positive and negative predictive values (PPV, NPV) were determined. McNemar’s test was used to assess whether there was a significant difference between the SLE-DAS and SLEDAI-2K sensitivity and specificity for clinically meaningful changes.
External validation
The external validation was performed using data from patients enrolled in the Padova Lupus Cohort. Construct validity was assessed through Spearman’s correlation between SLE-DAS, SLEDAI-2K and PGA at last follow-up visit. Considering the last two assessments in the follow-up period, patients with clinically meaningful worsening or improvement were identified. ROC curves were used to evaluate the performance of SLE-DAS and SLEDAI-2K in detecting clinically meaningful changes. McNemar’s test was used to assess whether there was a significant difference between the SLE-DAS and SLEDAI-2K sensitivity and specificity for clinically meaningful changes.
Patients with more than two follow-up visits and regular follow-up (defined as less than 12 months between visits) were included in the analysis of the ability of adjusted mean of SLE-DAS (amSLE-DAS) and adjusted mean of SLEDAI-2K (amSLEDAI) to predict damage accrual. The amSLE-DAS and amSLEDAI were determined for each patient and each visit as the area under the curve (AUC) of the SLE-DAS and SLEDAI-2K values over time, divided by the total length of the time period until that visit.24 Cox regression with time-dependent covariates25 was applied to assess the ability of the amSLE-DAS and amSLEDAI to predict damage accrual (ΔSDI≥1 during follow-up). The impact of each one of these variables in the hazard of damage was measured adjusting for other damage risk factors.26 Two models were estimated, one with amSLE-DAS (model 1) and another with amSLEDAI (model 2). The amSLE-DAS and amSLEDAI models were compared using Akaike information criterion (AIC) and Bayesian information criterion (BIC). The likelihood ratio test was applied to evaluate the importance of amSLE-DAS in a model which already has the information given by amSLEDAI and vice versa. Non-rejection of the null hypothesis indicates that the variables not included in the reduced model do not contribute significantly to predict the outcome.
IBM SPSS Statistics, V.24, and R software, V.3.1.2 were used, and p<0.05 was considered statistically significant.27
Results
Characteristics of the patients included
A total of 520 patients with SLE were included: 324 in the derivation cohort and 196 in the external validation cohort. Baseline characteristics of the patients included are presented in table 2.
Baseline characteristics of the patients included (n=520)
Determination of item weights and derivation of the SLE-DAS
Swollen joints, high urinary protein excretion, low platelet count, low white cell count, localized and generalized lupus rash, alopecia, mucosal membrane lesions, mucocutaneous vasculitis, low complement and increased levels of anti-dsDNA were observed in more than 10 patients and were included in the multivariate linear regression model. Urinary protein excretion, platelet count and white blood cell count were best modelled using the natural logarithmic function. These clinical and laboratory features were significantly associated with disease activity (as defined by PGA) in the multivariate model, and the R2 of the model was 0.91.
The remaining variables, neuropsychiatric involvement, systemic vasculitis, cardiac/pulmonary involvement, myositis, serositis and haemolytic anaemia were attributed discretionary weights, by expert decision as described above. These weights were incorporated into the final model.
The final SLE-DAS instrument is the sum of all the 17 weighted items (figure 1). The weighted scores of all these items and the detailed definition of each one can be found in the online supplementary table S1 and online supplementary table S2, respectively.
The final formula of Systemic Lupus Erythematosus Disease Activity Score (SLE-DAS). Alopecia, presence of alopecia (1=Yes, 0=No); Arthitis, presence of arthritis (1=Yes, 0=No); CardioPulm, presence of cardiac/pulmonary involvement (1=Yes, 0=No); GeneralRash, presence of generalized cutaneous rash (1=Yes, 0=No); Haemolytic, presence of haemolytic anaemia (1=Yes, 0=No); HypoC, presence of hypocomplementaemia (1=Yes, 0=No); IncreasedAnti-dsDNA, increased anti-dsDNA levels (1=Yes, 0=No); Leuc, leucopenia <3x10^9/L (1=Yes, 0=No); LeucCount, leucocyte count (10^9/L); LocalRash, presence of localized cutaneous rash (1=Yes, 0=No); MucocutVasculitis, presence of mucocutaneous vasculitis (1=Yes, 0=No); MucosalUlcers, presence of mucosal ulcers (1=Yes, 0=No); Myositis, presence of myositis (1=Yes, 0=No); Neuropsych, presence of neuropsychiatric involvement (1=Yes, 0=No); PlatCount, platelet count (10^9/L); PProt, proteinuria >500 mg/24 hours (1=Yes, 0=No); Prot, proteinuria (mg/24 hours); Serositis, presence of serositis (1=Yes, 0=No); SJC, swollen joint count (0–28); SystemicVasc, presence of systemic vasculitis (1=Yes, 0=No); Thromb, thrombocytopenia <100x10^9/L (1=Yes, 0=No).
Internal validation of SLE-DAS
In the derivation cohort, SLE-DAS score was highly correlated with PGA, as expected (rs=0.975, p<0.0005), and with SLEDAI-2K (rs=0.94, p<0.0005) at the last follow-up visit. Furthermore, SLEDAI-2K was also strongly correlated with PGA (rs=0.973, p<0.0005).
The performance of SLEDAI-2K and SLE-DAS to discriminate a clinically meaningful improvement or worsening (both defined as ΔPGA≥0.3) was assessed by ROC analysis. SLE-DAS variation had a higher discriminative performance (AUC=0.927 (95% CI 0.885 to 0.969, p<0.0005) than SLEDAI-2K (AUC=0.787 (95% CI 0.718 to 0.857), p<0.0005) to detect a clinically meaningful improvement (figure 2). Variations in SLE-DAS also presented higher discriminative ability than SLEDAI-2K to detect a clinically meaningful worsening, with AUCs of 0.994 (95% CI 0.988 to 1.000, p<0.0005) and 0.914 (95% CI 0.87 to 0.959, p<0.0005), respectively (figure 2). A cut-off value of 1.72 points in the ΔSLE-DAS was identified as the optimal discriminant for either a PGA increase (worsening) or decrease (improvement).
Receiver operating characteristic (ROC) curves comparing the performance of Systemic Lupus Erythematosus Disease Activity Score (SLE-DAS) and Systemic Lupus Erythematosus Disease Activity Index 2000 (SLEDAI-2K) to detect a clinically meaningful improvement (A, C) and worsening (B, D) in SLE disease activity. A and B correspond to the ROC curves of the derivation cohort and C and D correspond to the ROC curves of the external validation cohort.
A variation in SLE-DAS≥1.72 had a higher sensitivity to detect a clinically meaningful change in SLE disease activity compared with a SLEDAI-2K≥4 variation: for improvement (82.1% vs 44.8%, p<0.0005) and for worsening (93.1% vs 46.6%, p<0.0005), with similar specificities (96%–100%). Estimated sensitivities, specificities, PPV and NPV using cut-offs of decrease and increase in SLE-DAS≥1.72 and SLEDAI-2K≥4 points are shown in table 3.
Performance of SLE-DAS and SLEDAI-2K to detect clinically significant change in SLE disease activity
External validation of SLE-DAS
In the external validation cohort, SLE-DAS score was strongly correlated with PGA (rs=0.875, p<0.0005) and SLEDAI-2K (rs=0.943, p<0.0005) at the last follow-up visit. SLEDAI-2K was also correlated with PGA (rs=0.839, p<0.0005).
For a clinically meaningful improvement, in ROC analysis, a change in SLE-DAS had a higher performance (AUC=0.938 (95% CI 0.852 to 1.000), p<0.0005) than SLEDAI-2K (AUC=0.807 (95% CI 0.681 to 0.933), p<0.0005) (figure 2). For a clinically meaningful worsening, changes in SLE-DAS and SLEDAI-2K had an AUC of 0.998 (95% CI 0.994 to 1.000, p<0.0005) and 0.928 (95% CI 0.855 to 1.000, p<0.0005), respectively (figure 2).
A variation in SLE-DAS≥1.72 points showed a higher sensitivity to detect a clinically meaningful improvement and worsening in SLE disease activity than a change in SLEDAI-2K≥4 (89.5% vs 47.4% and 95.5% vs 59.1%, respectively, p=0.008 for both), maintaining similar specificities. Estimated sensitivities, specificities, PPV and NPV using cut-offs of increase or decrease in SLE-DAS≥1.72 and SLEDAI-2K≥4 points are reported in table 3.
A total of 192 patients had at least two visits with regular follow-up and were included in the analysis of the predictive value of amSLE-DAS and amSLEDAI for damage accrual. During a mean follow-up of 5 years, 44 (22.9%) patients had damage accrual. The amSLE-DAS and amSLEDAI were identified as independent predictors of damage accrual in the multivariate Cox regression analysis (table 4).
Multivariate Cox regression analysis for damage accrual in the validation cohort
Three multivariate Cox regression models for damage accrual over follow-up were estimated: model 1 with amSLE-DAS and model 2 with amSLEDAI, both variables being considered as time-dependent covariates; and model 3 with amSLE-DAS and amSLEDAI as time-dependent covariates. Gender, disease duration, age at diagnosis, presence of lupus nephritis, prednisolone dose and immunosuppressants at baseline were also included in these models as potential confounders (table 4). Both amSLE-DAS and amSLEDAI are significant predictors of damage, but model 1 (with amSLE-DAS) presents better fit than model 2 (with amSLEDAI), as it has lower values of AIC (377.183 vs 385.015, respectively) and BIC (389.673 vs 397.504, respectively). Furthermore, the likelihood ratio test comparing model 1 (with amSLE-DAS) with model 3 (containing both amSLE-DAS and amSLEDAI) yields a p value of 0.694, which indicates that there is no predictive advantage in including amSLEDAI when the model already considers amSLE-DAS information. On the contrary, the comparison of the model 3 with model 2 (containing only amSLEDAI) leads to the rejection of the null hypothesis (p=0.0047), which means that the prediction of damage improves significantly if we include amSLE-DAS in the model which already has amSLEDAI. Taken together, these analyses demonstrate that disease activity measured with SLE-DAS over time has a higher predictive value for damage accrual than SLEDAI-2K.
Discussion
SLE-DAS is a continuous global score of disease activity in SLE, designed to more accurately measure disease activity of SLE and to better assess its changes over time. SLE-DAS presented a good construct validity, with very high correlations with both SLEDAI-2K, a validated instrument, and PGA, both in the derivation and in the validation cohorts. Criterion validity was demonstrated by the predictive value of SLE-DAS for damage accrual, which was better than that of SLEDAI-2K.
SLE-DAS performs much better than SLEDAI-2K in terms of sensitivity to detect a clinically meaningful change, while presenting a high specificity. Such a performance can have major implications in the interpretation of clinical trials applying the disease activity as the primary endpoint, and in daily clinical practice, where SLE-DAS could provide robust guidance for treatment in the individual patient.
The assessment of disease activity in SLE is a very challenging task due to the heterogeneity of the disease manifestations and the difficulty of incorporating in a single instrument all the potentially relevant variables and their relative weights in assessing severity.28 Using a purely dichotomous approach to score variables, without grading the intensity of their change, makes the currently available tools insensitive to relevant changes, both at individual and group levels.29 These limitations underlie the difficulties that the scientific community has faced in establishing a defined clinical response as the outcome measure in randomised controlled trials or in defining remission and LDA as the targets in treat-to-target strategies.28 30 Actually, the current instruments cannot separate remission from LDA as we do with DAS28 in rheumatoid arthritis.31 These observations led us to the concept that a continuous measure of disease activity was indispensable to appropriately value change and to define cut-offs for remission, LDA and moderate or high disease activity.
In the absence of a validated gold standard, we considered that measuring SLE activity should consider all variables that have deserved acceptance by the scientific community so far and should be modelled on experienced physicians, whose judgement is validated by high-quality health outcomes documented in their patient cohorts.26 31 We started by acknowledging that in clinical practice the judgement of disease activity in a patient with SLE is based on a combination of laboratory and clinical features as well as the physician overall impression of the patient status (scored in PGA).28 The derivation of the SLE-DAS was modelled on the PGA and hence it was of utmost importance to ensure the accuracy of an expert clinician. Its construct validity was demonstrated in an external cohort with the PGA scored by a different expert clinician from another country and, most importantly, SLE-DAS was a better predictor of damage accrual than SLEDAI-2K, which argues for the objectivity of a better SLE-DAS performance.
In the SLE-DAS, the four parameters scored as continuous variables (arthritis, proteinuria, leucopenia and thrombocytopenia) are major drivers of its increased sensitivity to change. In addition, we expect that our new scoring algorithm and item redefinition, reduced from 24 (in SLEDAI) to 17 (in SLE-DAS), can also contribute to the improved performance of SLE-DAS. At least in some patients, it can be of major impact the inclusion of potentially severe features absent from SLEDAI (haemolytic anaemia, gastrointestinal and cardiopulmonary involvement), which were included in the current definitions of lupus LDA and remission.10 12 13 Assembling all these items into a quantitative disease activity index provided us with a sensitive and standardised instrument to assess disease activity in SLE.
Limitations of our study include the use given to PGA, given its subjective nature. However, PGA is considered the gold standard to assess SLE disease activity and has received renewed interest with the recent development of the Lupus Foundation of America-Rapid Evaluation of Activity, an index based on the assessment of a PGA for each organ system.32 Furthermore, in the current study, PGA was scored always by the same senior rheumatologist over the follow-up, its validity being supported by a high correlation with SLEDAI-2K in both cohorts. Additionally, the SLE-DAS was highly correlated with SLEDAI-2K and PGA in the internal and external validation cohort, supporting that PGA was consistently scored. Another important limitation refers external validation being performed, so far, in only one centre and mainly in Caucasian patients. Further studies are needed to confirm these results in multicentric, multiethnic cohorts. The prospective assessment of data at each clinical visit with consistently scored SLEDAI-2K, PGA and SDI, the internal and external validation including the damage accrual prediction are important strengths of our study.
In conclusion, SLE-DAS has good construct validity, a much higher discriminative power to detect clinically meaningful changes in SLE disease activity and a higher performance in predicting damage accrual, as compared with SLEDAI-2K. Future work from our group will focus in defining cut-off values for remission, LDA, moderate or high disease activity and developing an online calculator.
References
Footnotes
Handling editor Josef S Smolen
Contributors DJ, JAPS, AD and LSI contributed to the conception and design of the project, analysis and interpretation of data, and drafting and critical revision of the manuscript. DJ and LSI also contributed to patient follow-up at CHUC Lupus Cohort. AM and CH contributed to the design of the project, statistical analysis and interpretation of data, and critically revised the manuscript. MZ, ML and LI contributed to patient follow-up, and analysis and interpretation of data, and critically revised the manuscript. LSI and LI scored the disease measures in CHUC Lupus Cohort and Padova Lupus Cohort, respectively.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Obtained.
Ethics approval This study was approved by the Ethics Committee of the ‘Centro Hospitalar e Universitário de Coimbra’, Coimbra, Portugal and ‘Azienda Ospedaliera-Università degli Studi di Padova’, Padova, Italy. Informed consent was obtained from all patients before any study procedures.
Provenance and peer review Not commissioned; externally peer reviewed.