Article Text

Extended report
Prediction of cardiovascular risk in rheumatoid arthritis: performance of original and adapted SCORE algorithms
  1. E E A Arts1,
  2. C D Popa1,2,
  3. A A Den Broeder3,
  4. R Donders4,
  5. A Sandoo5,
  6. T Toms5,
  7. S Rollefstad6,
  8. E Ikdahl6,
  9. A G Semb6,
  10. G D Kitas5,
  11. P L C M Van Riel2,7,
  12. J Fransen1
  1. 1Department of Rheumatology, Radboud University Medical Center, Nijmegen, The Netherlands
  2. 2Department of Rheumatology, Bernhoven Hospital, Uden, The Netherlands
  3. 3Department of Rheumatology, Sint Maartenskliniek, Nijmegen, The Netherlands
  4. 4Department of Epidemiology, Biostatistics and Health Technology Assessment, Radboud University Medical Center, Nijmegen, The Netherlands
  5. 5Department of Rheumatology, Dudley Group NHS Foundation Trust, Dudley, UK
  6. 6Preventive Cardio-Rheuma Clinic, Diakonhjemmet Hospital, Oslo, Norway
  7. 7Radboud Institute for Health Sciences, IQ healthcare, Radboud University Medical Center, Nijmegen, The Netherlands
  1. Correspondence to E E A Arts, Department of Rheumatology, Radboud University Medical Center, Nijmegen 6500 HB, The Netherlands; elke.arts{at}radboudumc.nl

Abstract

Objectives Predictive performance of cardiovascular disease (CVD) risk calculators appears suboptimal in rheumatoid arthritis (RA). A disease-specific CVD risk algorithm may improve CVD risk prediction in RA. The objectives of this study are to adapt the Systematic COronary Risk Evaluation (SCORE) algorithm with determinants of CVD risk in RA and to assess the accuracy of CVD risk prediction calculated with the adapted SCORE algorithm.

Methods Data from the Nijmegen early RA inception cohort were used. The primary outcome was first CVD events. The SCORE algorithm was recalibrated by reweighing included traditional CVD risk factors and adapted by adding other potential predictors of CVD. Predictive performance of the recalibrated and adapted SCORE algorithms was assessed and the adapted SCORE was externally validated.

Results Of the 1016 included patients with RA, 103 patients experienced a CVD event. Discriminatory ability was comparable across the original, recalibrated and adapted SCORE algorithms. The Hosmer–Lemeshow test results indicated that all three algorithms provided poor model fit (p<0.05) for the Nijmegen and external validation cohort. The adapted SCORE algorithm mainly improves CVD risk estimation in non-event cases and does not show a clear advantage in reclassifying patients with RA who develop CVD (event cases) into more appropriate risk groups.

Conclusions This study demonstrates for the first time that adaptations of the SCORE algorithm do not provide sufficient improvement in risk prediction of future CVD in RA to serve as an appropriate alternative to the original SCORE. Risk assessment using the original SCORE algorithm may underestimate CVD risk in patients with RA.

  • Cardiovascular Disease
  • Rheumatoid Arthritis
  • Disease Activity
View Full Text

Statistics from Altmetric.com

Introduction

In rheumatoid arthritis (RA), cardiovascular disease (CVD) morbidity and mortality are increased.1 ,2 Inflammation may contribute to the increased risk of CVD,3–10 suggesting that inflammatory markers might be incorporated in CVD risk prediction models for RA. For example, high-sensitivity C reactive protein (CRP) was included in the recently developed Reynolds Risk Score (RRS), which is therefore better able to classify premenopausal women into different risk groups.11 ,12 Similarly, disease activity measures, such as the 28-joints Disease Activity Score (DAS28), may also be useful in CVD risk algorithms for patients with RA. Additionally, inflammation may add and modulate traditional CVD risk factors.13 ,14 Existing weights attributed to each individual risk factor included in currently available CVD risk algorithms may therefore require adjustment. The use of other CVD-related parameters not incorporated in the present CVD risk algorithms, such as carotid artery intima-media thickness and/or the presence of atherosclerotic carotid plaques in these patients, may also be considered.15–17

Current CVD risk prediction models, for example, Framingham Risk Score (FRS) or the Systematic COronary Risk Evaluation (SCORE) algorithm, have been developed for use in the general population.18–20 Their performance in patients with RA appears to be suboptimal.21–23 Therefore, it has been suggested that CVD risk algorithms based solely on traditional risk factors may not be suited for use in the RA population. As a first step towards more accurate CVD risk prediction, it was proposed in the European League Against Rheumatism (EULAR) recommendations for CVD risk management in RA to apply a multiplication factor of 1.5 to the calculated CVD risk by SCORE in selected patients to enhance the risk estimates.24 However, recent studies have shown that this multiplication factor does not significantly improve CVD risk prediction in patients with RA.25–27 A disease-specific CVD risk algorithm could perhaps improve prediction of CVD risk in patients with RA.

In Europe, the SCORE algorithm is widely used. A logical next step would be to evaluate whether the SCORE algorithm can be adapted to more accurately estimate the risk of CVD in patients with RA. Therefore, the objectives of this study were (1) to adapt the SCORE algorithm with determinants of CVD risk in patients with RA and (2) to compare the performance of the modified SCORE calculator to the original SCORE risk algorithm with regard to CVD risk prediction in patients with RA.

Patients and methods

Study design and patients

Prospectively collected data from the Nijmegen early RA inception cohort were used for this study. Patients were included at diagnosis of RA (baseline) in the outpatient clinic of the Departments of Rheumatology of the Radboud University Medical Centre (since 1985) or the Maartenskliniek (since 1990) in Nijmegen, The Netherlands. Patients with RA who fulfilled the 1987 American College of Rheumatology (ACR; inclusion before 2010) or ACR/EULAR 2010 criteria (inclusion after 2010) for the classification of RA,28 with disease duration of <1 year, and who were disease-modifying antirheumatic drug naïve were included. All patients gave written informed consent. Patients with a history of confirmed CVD before RA diagnosis and patients with a follow-up <18 months were excluded. Patients were censored after 10 years of follow-up or at the time of the first CVD event. The 10-year risk estimates for patients with a follow-up <10 years at the time of censoring (30 September 2011) were adjusted proportionally according to the actual follow-up time and calculated as a proportion of 10 years.22 The SCORE algorithm for prediction of both fatal and non-fatal disease in the Dutch population was used.20

Baseline predictors of cardiovascular events

In addition to the traditional risk factors used in the SCORE algorithm, potential RA-specific predictors for CVD were collected. Baseline characteristics were retrieved from the cohort database including; age (years), gender (male/female), rheumatoid factor (RF) status, anti-cyclic citrullinated peptide (anti-CCP) status, DAS28, erythrocyte sedimentation rate (ESR; mm/hour), swollen joint count (SJC), tender joint count (TJC) and the patient Visual Analogue Score (VAS) for global disease activity, Health Assessment Questionnaire (HAQ) and CRP (mg/L). Data on traditional CVD risk factors at baseline were collected by medical chart and electronic patient file review, including current smoking status (Y/N), blood pressure (mm Hg), height (m), weight (kg), diabetes mellitus (Y/N), hypertension (Y/N) and family history of premature CVD (Y/N). Lipid levels were measured using serum from frozen samples collected at baseline. Non-fasting total cholesterol (TC) and high-density-lipoprotein cholesterol (HDL-c) concentrations (mmol/L) were measured using laboratory facilities of Russells Hall Hospital, Dudley, UK.

Primary outcome

The primary outcome was first CVD events (physician diagnosed fatal or non-fatal CVD events), which were retrieved by extensive review of medical charts and electronic patient files. We included the following CVD events: acute coronary syndrome (myocardial infarction and unstable angina pectoris), cerebral vascular accident (CVA) and heart failure (HF). Deaths due to CVD were verified from death certificates, provided by Statistics Netherlands,29 including deaths due to CVD and CVA but excluding cerebral haemorrhage and non-coronary cardiac death.

Statistical analysis

The analysis consisted of two phases: (1) the recalibration and adaptation of the SCORE algorithm in our cohort and (2) evaluation of the predictive performance of the recalibrated and adapted SCORE algorithms. For the recalibration of the SCORE algorithm, the regression coefficients (the weights) of the predictors originally included in SCORE (current smoking status, systolic blood pressure and TC:HDL-c ratio) were newly estimated by means of Cox-proportional hazards regression analysis. The SCORE algorithm as developed by Conroy et al18 is fit for use in patients aged ≤65 years for the prediction of fatal CVD. Van Dis et al20 recalibrated the SCORE for the prediction of fatal and non-fatal CVD in the Dutch population and for individuals both aged ≤65 and >65 years. The SCORE algorithm was incorporated in the most recent version of the Dutch national guideline for cardiovascular risk management and we used this version.30 For the adaptation of the SCORE algorithm, other potential predictors were added to the existing SCORE variables. Variables with a significance level p<0.1 in univariate Cox proportional hazards regression analysis were evaluated in a multivariate Cox proportional hazard regression analysis. In the multivariate analysis, traditional CVD risk factors included in the SCORE algorithm were predetermined to stay in the model and other new predictors were included at a deliberately more liberal p value of <0.2. The following potential risk factors were considered: body mass index (BMI; weight (kg)/height (m)2), hypertension (physician diagnosis), diabetes mellitus (type I and II), diastolic blood pressure (mm Hg), TC (mmol/L), HDL-c (mmol/L), low-density lipoprotein cholesterol (mmol/L), triglycerides (mmol/L), family history of premature CVD, RF status, anti-CCP status, DAS28, CRP, ESR (mm/hour), SJC, TJC, VAS and HAQ. All analyses were performed with CVD events as the dependent variable and follow-up time (time since RA diagnosis) as the time variable.

Discrimination, that is, the number of patients who are correctly grouped into the event and the non-event group, was tested using the concordance statistic (c-statistic) and the area under the receiver operating characteristic (ROC) curve.31 Calibration was assessed by comparing the agreement between the observed number of CVD events and the number of CVD events predicted by the SCORE algorithms, in deciles of predicted CVD risk. Hosmer–Lemeshow (H–L) test and calibration plots were used to assess model fit. Clinical relevance of the models was assessed based on the number of patients reclassified into another CVD risk group: <10% (low risk), 10%–20% (intermediate risk) and >20% (high risk).20 ,30 The analyses were performed using SPSS V.21.0. Missing values on individual variables were imputed using multiple imputations with five repetitions.

External validation

The performance of the original, recalibrated and adapted SCORE algorithms concerning the prediction of fatal and non-fatal CVD was also analysed in external cohorts, consisting of 400 patients with RA from the UK (DRACCO cohort) and 204 patients from a Norwegian cohort (EURIDISS/ORAreg).32 ,33

Results

A total of 141 patients with a follow-up time of <18 months or documented CVD prior to RA diagnosis were excluded, leaving 1016 patients for analysis (table 1). During follow-up, 103 first CVD events occurred, including cases of acute coronary syndrome (n=66), ischaemic stroke (n=26), HF (n=4) and CVD deaths (n=7). At the time of event or censoring, patients had a mean±SD disease duration of 7.8±3.5 years.

Table 1

Patient characteristics

Model development

When recalibrating the SCORE algorithm only the traditional CVD risk factors were included in the Cox-proportional hazard regression, changing the regression coefficients (weights) as a result. For the adaptation of the SCORE model new variables were added (see online supplementary tables S1 and S2). The following CVD risk factors were significant predictors: current smoking status, systolic blood pressure, TC:HDL ratio, BMI, diabetes mellitus at baseline, hypertension at baseline, high baseline DAS28 (cut-off point >5.1). The median (IQR) 10-year CV risk scores calculated by the original and the adapted SCORE algorithm were 9.1% (2.7%–26.6%) and 6.7% (1.6%–18.4%) respectively.

Discrimination

Discriminatory performance was comparable across the original, recalibrated and adapted SCORE algorithms, with an area under the curve of 0.78 (95% CI 0.74 to 0.82), 0.78 (0.74 to 0.83) and 0.80 (0.75 to 0.84), respectively. The corresponding ROC curves for the original and adapted SCORE are presented in figure 1. The c-statistic values were similar between 0.75 and 0.76 for all three algorithms.

Figure 1

Receiver operating characteristic (ROC) curves for the original Systematic COronary Risk Evaluation (SCORE) and the adapted SCORE algorithms. Area under the curve values were (95% CI) 0.78 (0.74 to 0.82) and 0.80 (0.76 to 0.84) for the original and adapted SCORE algorithm, respectively.

Calibration

Patients were grouped into deciles based on ascending predicted CVD risk. In each of these groups the observed number of CVD events was compared with the calculated (expected) risk for CVD events. It appeared that when using the original SCORE algorithm, the CVD risk was underestimated in the lower and middle deciles and was greatly overestimated in the top decile (figures 2 and 3A). The H–L test indicated a poor model fit with a p value of <0.001. Next, the recalibrated algorithm that included only the ‘reweighted’ traditional risk factors underestimated CVD risk across all deciles with a p value of <0.001 for the H–L test, also indicating poor model fit. Then, the adapted SCORE algorithm underestimated CVD risk in the lower and middle deciles and overestimated CVD risk in the highest decile, with a p value for the H–L test of 0.04 (figures 2 and 3B).

Figure 2

Calibration plots. Probabilities depicted at the y axis, and the predicted probabilities depicted at the x axis as calculated by the original Systematic COronary Risk Evaluation (SCORE) algorithm (A) and the adapted SCORE algorithm (B). A line was fitted between the observed and predicted probabilities of cardiovascular disease events using cubic spline.

Figure 3

Bar chart with the observed number of events (%) depicted at the y axis, and the deciles of predicted risk depicted at the x axis as calculated by the original Systematic COronary Risk Evaluation (SCORE) algorithm (A) and the adapted SCORE algorithm (B). CV, cardiovascular.

Reclassification of CVD risk

For clinical care, patients usually are classified into risk groups. The number of patients classified by the original SCORE and the adapted SCORE in each of the three risk groups (>10%, 10%–20% and >20% CV risk) is presented in tables 2 and 3. Although the clinically important 10%–20% risk group did increase in numbers for the adapted SCORE, not all patients with a CVD event were reclassified appropriately because their predicted risk remained too low. In total, six patients with an event were reclassified to the low-risk group by the adapted SCORE compared with seven patients who were correctly reclassified to a higher risk group. The adapted SCORE model did perform noticeably better reclassifying the non-event patients (ie, their predicted risk became appropriately lower). Overall, estimates of CVD risk by the original SCORE and the adapted SCORE algorithms were similar for the majority of patients and did not lead to a reclassification into another risk group for most patients (68%).

Table 2

Patients grouped per CVD risk category for the SCORE original and the adapted SCORE

Table 3

Reclassification of risk group allocation for patients when CVD risk was calculated using the original SCORE and the adapted SCORE algorithms

External validation

After exclusion of 93 patients with a history of CVD at baseline, 511 patients were included for external validation (see online supplementary table S3). Patients had a mean±SD follow-up of 7.5±2.2 years. A total of 26 first fatal or non-fatal CVD events were registered including 17 acute coronary events, five cases of stroke, two cases of HF and two cases of other CV death. Due to missing values on non-traditional CVD risk factors, 24 events were available for analysis related to the validation the adapted SCORE algorithm. Missing values ranged from 0% to 5% (DAS28). Discriminatory ability of the adapted SCORE algorithm was inferior to the original SCORE algorithm with an area under the curve of 0.76 (95% CI 0.68 to 0.84), 0.74 (0.66 to 0.83) for the original and adapted SCORE algorithm, respectively. The H–L test indicated a poor model fit for both models with a p value of <0.001.

Discussion

In RA, risk assessment by traditional CVD risk models such as SCORE appears to be suboptimal.21 ,22 ,27 Therefore, the aim of this study was to adapt the SCORE algorithm to improve the accuracy of CVD risk estimates in patients with RA, by changing the weights (recalibration) and by adding new variables (adaptation). Unfortunately, recalibration and adaptation of the SCORE algorithm with additional RA-specific CVD risk factors did not lead to major improvements in the accuracy of CVD risk prediction in patients with RA.

Several RA-specific predictors were considered and some of them, such as high DAS28 at baseline, showed significant predictive power. However, in the end the SCORE algorithm adapted with RA-specific predictors showed a rather modest improvement in discriminatory ability in comparison with the original SCORE. Furthermore, the adapted SCORE algorithm mainly improved overestimation of CVD risk in patients not getting CVD and in the highest risk groups. Overestimation of CVD risk may be harmful as patients receive unnecessary treatment. However, overestimation of CVD risk mostly affected intermediate-risk to high-risk patients, in which case overestimation would only reaffirm treatment indication and would not change the indication for treatment. Improvement of CVD risk estimates in these patients therefore is less important for clinical purposes. Much could be gained from improving the classification of patients with RA who later develop CVD (event cases) into higher risk groups so these patients become eligible for preventive treatment. The adapted SCORE does not show a significant improvement in this area, leaving undetected high-risk patients with RA at risk of being under treated. In general, underestimation of CVD risk in RA appears to be the main problem with the original SCORE as shown by us as well as by others.21 ,27 Similar results have been reported for other CVD risk calculators such as FRS.22 The FRS significantly underestimated CVD risk, especially in older patients and in patients with positive RF and persistently elevated ESR. This indicates that in RA disease severity and inflammation that are not accounted for in current CVD risk algorithms may play a role. However, our adaptation of the SCORE algorithm including these variables did not solve the issue.

It is not clear why CVD risk generally is underestimated in RA. One explanation may be that the atherosclerotic burden in patients with RA is not always mirrored by the SCORE or other risk calculators.23 ,25 ,34 In patients with RA, the SCORE risk estimates did not associate well with subclinical carotid atherosclerosis25 and patients with RA with high coronary artery calcification were infrequently assigned to be at elevated risk by the FRS or the RRS.23 Calcifications and plaques of coronary arteries also occurred in patients with RA classified as having a low CV risk (<1%) according to the SCORE.34 Therefore, in RA important subclinical atherosclerosis is not reflected in CVD risk when applying CV risk calculators.

This study has several limitations. For one, the external validation cohort consists of patients with both early and established RA from varying geographical areas. Conflicting results have been reported with regards to the onset of the increased CVD in patients with RA.24 ,35–38 It may prove to be difficult to develop a singular CVD risk algorithm that can be applied successfully in all RA populations across different countries. Furthermore, the baseline risk that was determined in the general population for the original SCORE was also included in the adapted algorithm and this may contribute to systematic underestimation of CVD risk by this algorithm. As the baseline risk for CVD is increased in an individual with RA compared with a healthy counterpart of similar age and sex,1 ,2 it may be necessary to adapt the baseline risk. However, although our cohorts are among the largest RA cohorts available with sufficient data on CVD risk factors and follow-up, the number of available patients with RA and CVD events of this individual cohort was still deemed insufficient to determine a reliable, robust baseline risk for patients with RA that can be extrapolated to other RA populations. Furthermore, the adapted SCORE algorithm developed in this study is a basic revision of the original SCORE including all traditional CVD risk factors already included in the original SCORE. Although this approach is suitable for smaller cohorts it also comes with restrictions in terms of predictor selection. The structure and format of the SCORE are largely maintained, regardless of whether this would provide the best fit for the RA population. Hypothetically, some of the well-established traditional risk factors that form the base of the CVD risk algorithms that are currently used may require replacement by other, stronger, RA-specific predictors of CVD risk. These predictors may be more sensitive to the subtle differences between low-risk and high-risk patients in the RA population. The results of this study showed a significant relationship between high baseline disease activity and CVD risk. This is in concurrence with other research.17 ,38

In conclusion, this study demonstrates that adaptations of the SCORE algorithm do not provide sufficient improvement in the calculation of the risk for future CVD events in patients with RA to serve as an appropriate alternative to the original SCORE. A larger cohort with a higher number of CVD events might be used to develop a RA-specific CVD risk algorithm taking into consideration other factors, most intuitively related to the disease pathogenesis and inflammation. Alternatively, additional investigations such as carotid ultrasound may provide a substantial improvement of correct classification of these patients, even when using the original SCORE. Future studies should address these hypotheses to shed more light into this matter and contribute to a more efficient CV risk management in RA, eventually decreasing CV morbidity and mortality in this population.

Acknowledgments

The authors thank Dr Ineke van Dis from the Dutch Heart Foundation who generously provided us with the syntax for the recent Dutch adaptation of the SCORE algorithm.

References

View Abstract

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Files in this Data Supplement:

Footnotes

  • Handling editor Hans WJ Bijlsma

  • Contributors All authors have given substantial contribution to the conception and design and/or analysis and interpretation of the data, have drafted and/or revised the manuscript critically for important intellectual content and given final approval of the version to be submitted for publication. EEAA and JF had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

  • Competing interests All authors have completed the Unified Competing Interests form at http://www.icmje.org/coi_disclosure.pdf. EEAA was partially funded by the Rheumatology Research University Nijmegen foundation. AGS has received speaker honoraria and/or consulting fee from Merck/Schering-Plough, Abbott, BMS, Pfizer/Wyeth, Genentec and Hoffman-La Roche. GDK has served as a consultant UCB and Astra-Zeneca, received honoraria from Abbott, UCB and Pfizer and received research grants from Pfizer. PLCMvR has received travel grants and advisory board fees from Abbott, Pfizer, Roche and UCB.

  • Ethics approval Approved by the responsible medical ethical committee, CMO Arnhem Nijmegen.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.