Article Text
PDF
Abstract
Objective To develop and validate a diagnostic score that aids in identifying macrophage activation syndrome (MAS) in patients with systemic juvenile idiopathic arthritis (sJIA).
Methods The clinical and laboratory features of 362 patients with sJIA-associated MAS and 404 patients with active sJIA without evidence of MAS were collected in a multinational collaborative project. Eighty percent of the study population was used to develop the score and the remaining 20% constituted the validation sample. A Bayesian Model Averaging approach was used to assess the role of each clinical and laboratory variables in the diagnosis of MAS and to obtain the coefficients of selected variables. The final score, named MAS/sJIA (MS) score, resulted from the linear combination of these coefficients multiplied by the values of each variable. The cut-off that best discriminated MAS from active sJIA was calculated by means of receiver operating characteristic (ROC) curve analysis. Score performance was evaluated in both developmental and validation samples.
Results The MS score ranges from −8.4 to 41.8 and comprises seven variables: central nervous system dysfunction, haemorrhagic manifestations, active arthritis, platelet count, fibrinogen, lactate dehydrogenase and ferritin. A cut-off value ≥−2.1 revealed the best performance in discriminating MAS from active sJIA, with a sensitivity of 0.85, a specificity of 0.95 and a kappa value of 0.80. The good performance of the MS score was confirmed in the validation sample.
Conclusion The MS score is a powerful and feasible tool that may assist practitioners in making a timely diagnosis of MAS in patients with sJIA.
- macrophage activation syndrome
- systemic juvenile idiopathic arthritis
- hemophagocytic syndrome
- diagnostic score
- still’s disease
Statistics from Altmetric.com
- macrophage activation syndrome
- systemic juvenile idiopathic arthritis
- hemophagocytic syndrome
- diagnostic score
- still’s disease
Key messages
What is already known about this subject?
Macrophage activation syndrome (MAS) is a potentially life-threatening complication of systemic juvenile idiopathic arthritis (sJIA). Timely diagnosis of MAS and prompt institution of appropriate treatment are fundamental and can be life-saving for the patient. However, early diagnosis of MAS is often difficult.
Two sets of diagnostic guidelines for MAS in sJIA have been proposed in the past, but none of them is considered sufficiently reliable. Although classification criteria for MAS complicating sJIA have been published in 2016, these criteria were developed to classify correctly patients included in research studies and clinical trials, but are not intended for use in the diagnosis of the syndrome in routine care.
What does this study add?
In validation analyses, the MAS/sJIA (MS) revealed a strong capacity to discriminate MAS from active sJIA without evidence of MAS.
How might this impact on clinical practice or future developments?
The MS score is a simple and feasible tool which may assist practitioners in timely identification of MAS in the setting of active sJIA. The MS score may prove applicable and useful for the detection of MAS also in adult-onset Still’s disease.
Introduction
Macrophage activation syndrome (MAS) is a potentially life-threatening complication of rheumatic disorders, which is encountered most frequently in systemic juvenile idiopathic arthritis (sJIA) and in its adult equivalent, adult-onset Still’s disease (AOSD).1 2 Its pathophysiologic hallmark is a hyperinflammatory reaction resulting from a highly stimulated but ineffective immune response, which results in a cytokine storm syndrome.3 4 The estimated prevalence of MAS in sJIA is around 10%, but increasing evidence suggests that subclinical forms of the syndrome may occur in up to 30%–40% of patients with active systemic disease.5 6
The cardinal clinical features of MAS are prolonged high fever, hepatosplenomegaly, generalised lymphadenopathy, central nervous system (CNS) dysfunction and haemorrhagic manifestations. Typical laboratory abnormalities include drop in blood cell lines, elevation of liver enzymes, triglycerides, lactate dehydrogenase and ferritin, and decreased levels of fibrinogen. Although hemophagocytosis is often seen on bone marrow examination, this finding may be absent, particularly in the initial stages of the syndrome.7
Because MAS is potentially fatal, timely diagnosis and prompt initiation of appropriate treatment are fundamental to avoid a deleterious outcome. However, there is no single feature that is specific for MAS, including hemophagocytosis.8 In addition, MAS can be difficult to distinguish from conditions that may be present with overlapping manifestations, such as flares of sJIA or systemic infections. The diagnostic challenges emphasise the utility of criteria that could aid physicians in identifying MAS in its earliest stages and in distinguishing it from confusable disorders.
Historically, two sets of criteria have been proposed for diagnosis of MAS: the 2004 diagnostic guidelines for hemophagocytic lymphohistiocytosis (HLH-2004)9 and the preliminary diagnostic guidelines for MAS complicating sJIA.10 However, although both guidelines are considered suitable for detecting MAS in sJIA, each was found to suffer from several limitations.11
Recently, an international collaborative effort has led to promulgate the 2016 classification criteria for MAS complicating sJIA.12 13 However, these criteria have been primarily proposed for use in clinical trials and research studies. Furthermore, evidence was found that they could not capture all instances of MAS seen in the routine clinical setting, particularly those with subtle onset or incomplete clinical expression.12 13 A recent systematic literature review has shown that MAS classification criteria may miss some episodes of MAS occurring in patients with sJIA under treatment with IL-1 and IL-6 blocking agents, owing to the substantial alterations in MAS features induced by these biologics.14
One of the reasons that could explain the inadequate performance of MAS classification criteria in diagnosing MAS in daily practice is the use in their development of a control group of patients with systemic infection. Because these patients had much less pronounced inflammatory laboratory features than the other control sample with active sJIA without evidence of MAS, the combination of the two groups might have ‘diluted’ the features of the control population and, hence, inflated the value of laboratory abnormalities needed to discriminate MAS patients from controls.
Against this background, the primary purpose of the present study was to develop and validate a weighted score for the diagnosis of MAS in patients with sJIA, named MAS/sJIA (MS) score, using the same data sets of the MAS classification criteria study, but excluding the control sample with systemic infection.
Methods
Study design and patient selection
Data of patients with sJIA-associated MAS and patients with active sJIA without evidence of MAS were collected in the context of the multinational collaborative effort that led to the development of the 2016 classification criteria for MAS complicating sJIA.12 13 The design, inclusion criteria and data collection procedures of this project have been described in detail previously.7 11–13 15
To be included in the study, patients with sJIA-associated MAS had to have sJIA according to International League of Associations for Rheumatology (ILAR) criteria16 and to have had an episode of MAS diagnosed and treated as such by the caring physician. Patients with active sJIA without MAS should also meet ILAR criteria for sJIA, but should not have any feature of MAS, including cytopenia, hepatitis, coagulopathy and hyperferritinemia. Data for patients with MAS were collected at onset of the syndrome (ie, at the time when the first signs or symptoms consistent with the syndrome were detected), whereas data for patients with active sJIA without MAS were collected at onset of sJIA or at the time of a disease flare.
CNS dysfunction was defined as the presence of lethargy, seizures, irritability, confusion, headache, mood changes or coma. Haemorrhagic manifestations were defined as the presence of petechiae, ecchymoses or purpura, mucosal or gastrointestinal bleeding or intravascular coagulation. Evidence of hemophagocytosis in the bone marrow aspirate was not assessed because this procedure is not routinely performed in patients with sJIA without MAS. The study protocol was approved by the Ethics Committee at each participating centre.
Development of the MS score
The MS score was constructed through the following steps:
Step 1
Creation of developmental and validation data sets
Eighty percent of patients enrolled in the study were assigned through random computer generation to the developmental data set, and the remaining 20% were assigned to the validation data set. The characteristics of patients with MAS and with active sJIA without MAS were compared by χ2 test (for categorical variables) or Mann-Whitney U test (for continuous variables). Missing values were imputed using a random forest imputation (R package missForest)17 The high rate (>60%) of missing values prevented the inclusion of D-dimer in the analyses.
Step 2
Selection of clinical and laboratory candidate variables
To assess the relative role of clinical and laboratory variables in the diagnosis of MAS, a Bayesian Model Averaging (BMA) approach was used. Bayesian methods are increasingly used in developmental research, especially to solve the problems of model selection. BMA is an extension of the Bayesian inference methods that consider both model and parameter uncertainty.18 The BMA approach combines weighted fitted values from multiple models to estimate the posterior distribution of the model parameters and provides posterior mean, posterior SD and the probability of a variable inclusion. An inclusion probability of 0.5 indicates 50% certainty that the variable should be included in the model, whereas an inclusion probability of 1 indicates 100% certainty.19 For the purpose of our analysis, we selected the factors with an inclusion probability greater than 0.8. BMA computation was performed using the R package BAS, with the default prior distributions.
Step 3
Creation of the MS score
The coefficients of the selected variables were obtained from the average of all the possible values included in all 2p developed models, where p was the number of variables taken into account. The MS score resulted from the linear combination of the coefficients, multiplied by the values of the corresponding variable. Higher scores indicated a greater probability of having MAS, whereas lower scores indicated a greater probability of having active sJIA without MAS.
Step 4
Performance and validation of the model
The cut-off value in the MS score that provided the best discrimination between MAS and active sJIA without MAS was calculated by means of receiver operator characteristic (ROC) curve analysis as the cut-off that maximised accuracy, computed according to Grenier.20 To evaluate the performance of the model, sensitivity, specificity, negative predictive value, positive predictive value, area under the curve (AUC) and kappa value were calculated for both developmental and validation samples.
All analyses were carried out using SAS software V.9.3 and R V.3.5.
Patient and public involvement
Patients and the public were not involved in the research because the project did not address any issues of specific interest to these stakeholders.
Results
Study population
A total of 766 patients were included in the study: 362 had sJIA-associated MAS and 404 had active sJIA without MAS. The comparison of demographic, clinical laboratory and histopathologic features between the two patient groups is shown in tables 1 and 2. The proportion of females and the age at onset of sJIA were comparable between patients with and without MAS. All clinical manifestations were more common in the MAS cohort than in the active sJIA group, with the exception of fever, which was present in an equal percentage of patients with both conditions, and of active arthritis, which was recorded more commonly in patients with active sJIA.
Demographic, clinical and histopathologic features of patients with MAS and with active sJIA without MAS*
Values of laboratory tests in patients with MAS and with active sJIA without MAS*
All laboratory values were more abnormal in patients with MAS when compared with patients without MAS, with the exception of the erythrocyte sedimentation rate, which was higher in the latter group of patients. Overall, the values from laboratory tests in patients with MAS reflected the typical changes that are known to occur in the syndrome. Bone marrow hemophagocytosis was looked for only in the MAS sample and was detected in 59.8% of patients who underwent a bone marrow aspirate.
Development of the MS score
The developmental dataset was composed of 613 patients, 285 with MAS and 328 with active sJIA without MAS; the remaining 153 patients (77 with MAS and 76 without MAS) were included in the validation data set. The developmental and validation samples were comparable for all demographic, clinical and laboratory features (see online supplementary table S1).
Supplemental material
The 23 candidate clinical and laboratory variables analysed yielded in more than 8 million (223) logistic regression models. With the Bayesian approach, all possible combinations of predictors were assumed to be equally likely. The best 20 models obtained through the BMA are presented in figure 1. To implement the Bayesian approach, we computed the posterior model probability for all possible models using a ‘Monte Carlo Markov Chain+BAS’ method and selected all the variables with a posterior probability greater than 0.8 (table 3). The beta coefficients and the 95% CI of the selected variables that compose the final model of the MS score are shown in table 4, together with the formula that should be used to calculate the MS score, which is the following: CNS involvement×2.44+haemorrhagic manifestations×1.54+arthritis×(−1.30)+platelet count×(−0.003)+lactic dehydrogenase×0.001+fibrinogen×(−0.004)+ferritin×0.0001. For clinical features (CNS involvement, haemorrhagic manifestations and active arthritis), a score of 0 or 1 is placed in the formula, depending on whether they are absent or present, respectively. Laboratory tests are included in the formula with their observed value in the unit reported in table 4. The MS score resulted in a continuous variable ranging from −8.4 to 41.8. The AUC of the model was 0.95 (95%CI: 0.93 to 0.97). The probability of a diagnosis of MAS assessed through the application of the MS score as continuous measure is presented in the online supplementary table S2. An excel file that aids in the calculation of the MS score is provided in the online supplementary file S3.
Supplemental material
Supplemental material
Best 20 models of diagnostic criteria obtained through the Bayesian Model Averaging approach. The models are ranked on the x-axis by log posterior odds values, with performance decreasing from the left to the right. Variables included in each model are coloured, with the colours being related to the log posterior probability. Variables not included in the model are coloured in black.
Posterior probability of each demographic, clinical and laboratory variable to enter the logistic regression models with the best performance in the discrimination of patients with MAS from patients with sJIA without MAS (N=613)
Best-fitted model with the best combination of clinical and laboratory variables and β-coefficients used for the calculation of the MS score
The ROC curve analysis identified a score of −2.1 as the cut-off that provided the best discrimination between MAS and active sJIA without MAS, with a score ≥−2.1 being indicative of MAS. The application of this cut-off in the developmental sample yielded a sensitivity of 85%, a specificity of 95% an AUC of 0.95 and a kappa value of 0.80 (table 5). The statistical performance of the MS score in the validation sample (n=153) was slightly better than in the developmental cohort: sensitivity 89%, specificity 99%, AUC 0.97 and kappa value equal to 0.87 (table 5).
Statistical performance of the MS score in developmental and validation samples
Although fever was not comprised among the variables selected to compose the MS score, we considered that nearly all patients in the two samples used to validate the MS score had fever and that fever is a cardinal clinical manifestation of MAS. We, therefore, included the presence of fever as mandatory criterion for the diagnosis of MAS (table 4).
The validity of the score was further scrutinised by evaluating its ability to discriminate patients with a diagnosis of MAS (n=91) or non-MAS (n=401) confirmed by the experts who participated in the consensus conference that led to the development of the 2016 classification criteria for MAS complicating sJIA.12 13 The application of the MS score on this cohort yielded a sensitivity of 97%, a specificity of 97%, an AUC of 0.97 and kappa value of 0.89. The good performance of MS score was also confirmed in the population of patients who developed MAS at onset of sJIA (n=77) (sensitivity 78%, specificity 96%, AUC 0.93 and kappa value 0.74).
The homogeneity of developmental and validation samples is underscored by the violin plot for distribution of MS scores (see online supplementary figure S4). A violin plot and a dotplot for distribution of MS scores in sJIA and MAS samples (shown in online supplementary figures S5 and S6, respectively) demonstrate that the two populations constitute a continuum, although they are separated in two clusters by the MS score value.
Supplemental material
Discussion
We have developed a diagnostic score aimed to aid physicians in timely detection of MAS in patients with active sJIA. The construction of the tool was based on a large data set of patients with sJIA-associated MAS and with active sJIA without evidence of MAS recruited in tertiary care hospitals located in 33 countries in five continents.7 11 Owing to its large size and wide geographic origin, the study population is likely representative of patients with sJIA with and without MAS seen in most referral centres worldwide. The MS score comprises seven clinical and laboratory variables selected and weighted through a BMA approach on the basis of the strength of their association with the diagnosis of MAS. The score ranges from −8.4 to 41.8, and the diagnosis of MAS is more likely when the score is greater than −2.1. The discriminative ability of the MS score was excellent in both developmental and validation data sets. The slightly superior performance of the score in the validation sample than in the developmental cohort could be partly due to the better tendency of the laboratory variables included in the validation sample to follow the same trend of the laboratory components of the MS score.
Fever was not comprised among the variables selected to compose the MS score because its frequency was similar in patients with and without MAS, as it was recorded in nearly all cases in both samples. However, because fever is a cardinal clinical manifestation of MAS, was the mostly highly ranked clinical feature of MAS in a Delphi survey conducted among international paediatric rheumatologists21 and was considered a prerequisite for the classification of MAS by the expert panel that devised the 2016 classification criteria for MAS complicating sJIA,12 13 we have added the presence of fever as mandatory criterion for the diagnosis of MAS to the variables included in the MS score.
Of the other typical clinical manifestations of MAS, only CNS dysfunction and haemorrhagic manifestations were incorporated in the MS score. CNS dysfunction and haemorrhagic manifestations revealed strong discriminative properties and were assigned the highest weights. The absence of arthritis, which is one of the main clinical features of active sJIA, was also a risk factor for MAS. This finding is in keeping with the notion that the onset of MAS in sJIA is often accompanied by a paradoxical improvement of the inflammatory activity of the underlying disease.1 Note that a recent analysis of cytokine profiles of patients with s-JIA revealed that two distinct subsets could be identified on the basis of their serum IL-6 and IL-18 levels, one prone to arthritis and another susceptible to MAS.22 Among laboratory biomarkers, platelet count, lactate dehydrogenase, fibrinogen and ferritin are part of the MS score.
The composition of the MS score is different in several respects from the two previous criteria sets specifically designed for MAS in sJIA. Only four of the seven variables included in the preliminary diagnostic guidelines for MAS complicating sJIA10 are incorporated in the MS score. Aspartate aminotransferase and triglycerides, which are part of the 2016 classification criteria for MAS complicating sJIA, are not included in the MS score. Another major difference between the MS score and previous diagnostic or classification sets for MAS is that no threshold value is provided for laboratory tests, but the contribution of laboratory variables is computed by multiplying their observed value by the coefficient obtained through statistical procedures.
Some caveats should be taken into account in interpreting our analysis. We should acknowledge that the diagnostic categories of sJIA with MAS and active sJIA without MAS were based on caring physician’s judgement, which could be affected by level of experience. However, the discriminative performance of the score was similar when the analysis was restricted only to patients who had the diagnosis of MAS confirmed by a consensus (>80%) of experts. Some important biomarkers of MAS, such as sCD25, sCD163 IL-18 and CXCL9 levels and natural killer cell activity, could not be assessed owing to their unavailability in both patient samples. However, these biomarkers are not routinely assessed, nor are they timely, in most paediatric rheumatology centres. Likewise, the diagnostic ability of bone marrow hemophagocytosis could not be evaluated as information about this procedure was missing for all patients with sJIA without MAS. Due to the small number of patients under biological treatments in the database, the MS score could not be tested in a subgroup of patients who had MAS under therapy with IL-1 and IL-6 inhibitors.
In conclusion, the MS score is a powerful tool that may facilitate timely detection of MAS in the setting of active sJIA. This score is feasible and easily applicable in routine clinical practice, which should result in its widespread acceptance and use. Future assessments at the bedside can be enhanced and made easier by developing a phone/web application. The validity of the score should be examined in registry cohorts of sJIA and by assessing its correlation with already available or blinded case review of the presence/absence of MAS. Importantly, the MS score is not intended for use in paediatric rheumatic disorders other than sJIA, such as childhood-onset systemic lupus erythematosus, juvenile dermatomyositis or Kawasaki disease. However, considering that sJIA and AOSD are nowadays thought to constitute the same disease entity occurring at different ages,23 24 it is likely worth testing the capacity of the MS score to capture MAS in adult patients with AOSD.
Supplemental material
Supplemental material
Supplemental material
Acknowledgments
The authors thank the following physicians who contributed to the study by including their patients’ data: Mario Abinun, MD (Newcastle, UK); Amita Aggarwal, MD (Lucknow, India); Jonathan Akikusa, MD (Melbourne, Australia); Nuray Aktay Ayaz, MD (Istanbul, Turkey); Sulaiman Al-Mayouf, MD (Riyadh, Saudi Arabia); Maria Alessio, MD (Naples, Italy); Jordi Aton, MD (Barcelona, Spain); Maria Teresa Apaz, MD (Cordoba, Argentina); Itziar Astigarraga, MD (Bilbao, Spain); Tadej Avcin (Ljubljana, Slovenia); Patrizia Barone, MD (Catania, Italy); Bianca Bica, MD (Rio de Janeiro, Brazil); Isabel Bolt, MD (Berne, Switzerland); Luciana Breda, MD (Chieti, Italy); Rolando Cimaz, MD (Florence, Italy); Fabrizia Corona, MD (Milan, Italy); Ruben Cuttica, MD (Buenos Aires, Argentina); Gianfranco D’Angelo, MD (Ancona, Italy); Zane Davidsone, MD (Riga, Latvia); Carmen De Cunto, MD (Buenos Aires, Argentina); Jaime De Inocencio, MD (Madrid, Spain); Erkan Demirkaya, MD (Ankara, Turkey); Eli M. Eisenstein, MD (Jerusalem, Israel); Sandra Enciso, MD (Mexico City, Mexico); Graciela Espada, MD (Buenos Aires, Argentina); Romina Gallizzi, MD (Messina, Italy); Maria Luz Gamir, MD (Madrid, Spain); Yi-Jin Gao, MD (Shanghai, China); Thomas Griffin, MD (Charlotte, NC); Alexei Grom, MD (Cincinnati, OH); Saad Hashad, MD (Tripoli, Libya); Teresa Hennon, MD (Buffalo, NY); Gerd Horneff, MD (St. Agustin, Germany); Zeng Huasong, MD (Guangzhou, China); Norman Ilowite, MD (New York, NY); Antonella Insalaco, MD (Rome, Italy); Maka Ioseliani, MD (Tbilisi, Georgia); Michael Jeng, MD (Stanford, CA); Agneza Marija Kapovi ́c, MD (Zagreb, Croatia); Ozgur Kasapcopur, MD (Istanbul, Turkey); Toshiyuki Kitoh, MD (Nagakute, Japan); Isabelle Kone-Paut, MD (Paris, France); Sheila Knupp Feitosa de Oliveira, MD (Rio de Janeiro, Brazil); Bianca Lattanzi, MD (Ancona, Italy); Kai Lehmberg, MD (Hamburg, Germany); Lepore Loredana, MD (Trieste, Italy); Caifeng Li, MD (Beijing, China); Jeffrey M. Lipton, MD (New York, NY); Silvia Magni-Manzoni, MD (Rome, Italy); Rosa Merino, MD (Madrid, Spain); Paivi Miettunen, MD (Calgary, Canada); Velma Mulaosmanovic, MD (Sarajevo, Bosnia and Herzegovina); Clarissa Nassif, MD (Belo Horizonte, Brazil); Susan Nielsen, MD (Copenhagen, Denmark); Seza Ozen, MD (Ankara, Turkey); Priyankar Pal, MD (Kolkata, India); Sampath Prahalad, MD (Salt Lake City, UT); Raju Khubchandani, MD (Mumbai, India); Ingrida Rumba-Rozenfelde, MD, PhD (Riga, Latvia); Ricardo Russo, MD (Buenos Aires, Argentina); Claudia Saad Magalh ̃aes, MD (Botucatu,Brazil); Wafaa Mohamed Saad Sewairi, MD (Riyadh, Saudi Arabia); Helga Sanner, MD (Oslo, Norway); Susan Shenoi, MD (Seattle, WA); Valda Stanevicha, MD (Riga, Latvia); Gary Sterba, MD (Miami, FL); Kimo C. Stine, MD (Little Rock, AK); Gordana Susic, MD (Belgrade, Serbia); Flavio Sztajnbok, MD (Rio de Janeiro, Brazil); Syuji Takei, MD (Kagoshima City, Japan); Ralf Trauzeddel, MD (Berlin, Germany); Elena Tsitsami, MD (Athens, Greece); Yosef Uziel, MD (Kfar Saba, Israel); Olga Vougiouka, MD (Athens, Greece); Carol A. Wallace, MD (Seattle, WA); Lehn Weaver, MD (Philadelphia, PA); Jennifer E. Weiss, MD (Hackensack, NJ); Sheila Weitzman, MD (Toronto, Canada); Carine Wouters, MD (Leuven, Belgium); Nico Wulffraat, MD (Utrecht, The Netherlands) and Mabruka Zletni, MD (Tripoli, Libya).
References
Footnotes
Handling editor Josef S Smolen
Contributors We confirm that all authors have contributed to the generation of the manuscript.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement All data relevant to the study are included in the article or uploaded as supplementary information.