Article Text
Abstract
Objectives To develop data-driven criteria for clinically inactive disease on and off therapy for juvenile dermatomyositis (JDM).
Methods The Paediatric Rheumatology International Trials Organisation (PRINTO) database contains 275 patients with active JDM evaluated prospectively up to 24 months. Thirty-eight patients off therapy at 24 months were defined as clinically inactive and included in the reference group. These were compared with a random sample of 76 patients who had active disease at study baseline. Individual measures of muscle strength/endurance, muscle enzymes, physician's and parent's global disease activity/damage evaluations, inactive disease criteria derived from the literature and other ad hoc criteria were evaluated for sensitivity, specificity and Cohen's κ agreement.
Results The individual measures that best characterised inactive disease (sensitivity and specificity >0.8 and Cohen's κ >0.8) were manual muscle testing (MMT) ≥78, physician global assessment of muscle activity=0, physician global assessment of overall disease activity (PhyGloVAS) ≤0.2, Childhood Myositis Assessment Scale (CMAS) ≥48, Disease Activity Score ≤3 and Myositis Disease Activity Assessment Visual Analogue Scale ≤0.2. The best combination of variables to classify a patient as being in a state of inactive disease on or off therapy is at least three of four of the following criteria: creatine kinase ≤150, CMAS ≥48, MMT ≥78 and PhyGloVAS ≤0.2. After 24 months, 30/31 patients (96.8%) were inactive off therapy and 69/145 (47.6%) were inactive on therapy.
Conclusion PRINTO established data-driven criteria with clearly evidence-based cut-off values to identify JDM patients with clinically inactive disease. These criteria can be used in clinical trials, in research and in clinical practice.
Statistics from Altmetric.com
Juvenile dermatomyositis (JDM) is a systemic autoimmune vasculopathy that primarily involves skin and muscles and is characterised by proximal muscle weakness and typical rashes, including Gottron's papules and heliotrope rash. While mortality has been greatly reduced to 2–3%,1 morbidity has increased with 70–80% of patients having cumulative damage after a mean of 8 years.2,–,5
With the advent of new therapies and treatment strategies for JDM,6 inactive disease has become a realistic therapeutic target in clinical practice.3 ,5 However, no formal criteria for clinically inactive disease for JDM are in place. Differences in previously used criteria have been noted between studies, and most of the criteria include generic concepts without clearly defined cut-off values or operational definitions.2 ,7,–,10
The aim of this project was to develop data-driven criteria of clinically inactive disease by analysing a large international prospective cohort of patients with JDM. The overall goal was to propose criteria which could have practical applicability in current clinical practice, research and in future clinical trials.
Methods
Database content
Centres of the Paediatric Rheumatology International Trials Organisation (PRINTO)11 prospectively collected data on clinical, laboratory and therapeutic modalities in consecutive patients who had probable/definite JDM,12 ,13 were aged <18 years and were in an active phase of their disease, defined as either the need to start corticosteroid therapy or a new immunosuppressive medication or to have a major increase in dose. Written or verbal informed consent/assent was obtained as per local requirements.
The database contains the six PRINTO JDM core set measures14 assessed longitudinally (0, 6, 12 and 24 months): (1) physician's global assessment of the patient's overall disease activity on a 10 cm visual analogue scale (MD-GLOVAS);15 (2) muscle strength/endurance on the Childhood Myositis Assessment Scale (CMAS)16,–,18 or manual muscle testing of eight muscle groups (MMT);19 (3) global disease activity assessment through the Disease Activity Score (DAS);20 (4) functional ability through the Childhood Health Assessment Questionnaire (C-HAQ);21,–,24 (5) parent's global assessment of the patient's overall well-being on a 10 cm VAS (Par GLOVAS);15 ,21 ,22 and (6) health-related quality of life using the physical summary score (PhS) of the Child Health Questionnaire (CHQ).22 ,25 Additional measures were the Myositis Disease Activity Assessment which combines the Myositis Disease Activity Assessment Visual Analogue Scale (MYOACT) including the physician evaluation of extramuscular activity (MD-ExtraMuscVAS) and muscle activity (MD-MuscVAS) and the Myositis Intention to Treat Activity Index;26 the psychosocial summary score (PsS) of the CHQ;22 ,25 serum muscle enzymes (creatine kinase (CPK), lactate dehydrogenase (LDH), aldolase, aspartate aminotransferase, alanine aminotransferase (ALT)),27,–,31 whose results were standardised as previously described;32 and the Myositis Damage Index.4 ,26 ,33 The scoring and content of all the instruments have been previously reported.14
Study design and inclusion criteria
To identify the features that were suitable as criteria for clinically inactive disease, we followed the classification approach. The purpose was to separate patients with inactive disease (reference sample) from patients with active disease (comparison sample), with high sensitivity and specificity. As shown in figure 1, the reference sample was represented by patients off therapy with an inactive disease status by definition at 24 months; in addition, patients were declared stable or improved with respect to the previous visit by the physician and/or the parents. In order to avoid selection biases, the comparison sample was represented by the baseline data (patient active by study inclusion criteria) of a computer-generated random list of patients who were still on therapy after 24 months of follow-up. Similar to the work done in juvenile idiopathic arthritis (JIA), we defined clinically inactive disease as a single point in time status with clinically and biologically quiescent disease that can be on/off therapy. When this criterion is met for at least 6 or 12 continuous months, this status is called clinical remission on therapy or off therapy, respectively.34 ,35
Validation process
Four steps were used for the analysis.
In Step 1 (cut-off selection) the reference sample off therapy at 24 months was described. The hypothesis was that the descriptive values for each variable considered would be representative of inactive disease status and able to discriminate inactive from active patients. Based on the descriptive statistics (mean±SD, minimum, 20th percentile, median, 80th percentile and maximum) of each variable, we then selected candidate cut-off values that could properly describe a patient as clinically inactive off therapy. The strategy was as follows: for those variables where the lowest value correlated with inactive disease (eg, muscle enzymes), we chose the median, 20th percentile or the minimum values. In contrast, those variables in which higher values correlate with disease inactivity (eg, CMAS, MMT), the median, 80th percentile or the maximum value was considered to be the best cut-off value. Additional candidate cut-off values were also derived from the literature.2 ,7 ,8 ,10
In Step 2 (accuracy measures) we evaluated the ability (accuracy) of the cut-off values for each variable to discriminate the reference sample from the comparison sample by calculating the sensitivity/specificity and Cohen's κ.36 The κ statistic according to Landis and Koch37 was categorised as follows: 0.01–0.2=slight; 0.21–0.4=fair; 0.41–0.6=moderate; 0.61–0.8=substantial; 0.81–1=almost perfect agreement. In this step we expected that only a few, if any, of the active patients in the comparison group would have values consistent with cut-offs selected as representative of the reference sample.
In Step 3 (inactive disease criteria testing) we tested 19 candidate criteria (combination of individual measures and related cut-off values) of inactive disease derived from the literature3 ,9 ,38,–,44 and an additional 35 criteria that, based on the results of Steps 1 and 2, were deemed clinically reasonable by the Steering Committee of the project (DL, AP, NR, AR, AM, PM, EP), by calculating their sensitivity/specificity and Cohen's κ.36 Moreover, in this step we expected that only a few, if any, of the patients in the comparison sample would fit the inactive disease criteria while most, if not all, of the patients in the reference sample would.
Finally, in Step 4 (criteria confirmation) we applied the top criteria selected in Step 3 to both the reference sample off therapy and the remaining patients still on therapy at 24 months (figure 1). In this analysis, among the patients still on therapy at 24 months, we identified the subgroup of patients who met the criteria of inactive disease on medication and differentiated them from those patients who did not meet the criteria of inactive disease and who were therefore considered to have still active JDM despite 24 months of therapy. Our hypothesis was that, if our top criteria selected in Step 3 allowed us to correctly identify the group with inactive disease on therapy at 24 months, the individual activity in this group should be quite similar to the reference sample.
Data were entered into an Access XP database and analysed with Excel XP (Microsoft), XLSTAT 6.1.9 Addinsoft, Statistica 6.0 (StatSoft Inc) and Stata 7.0 (Stata Corporation).
Results
Demographic characteristics
A total of 275 patients with active JDM were retrieved from the PRINTO database (figure 1); 168 (61%) were girls with a median age at disease onset of 7.2 years (IQR 4.3–10.2) and a median disease duration of 0.6 years (IQR 0.2–2.1).14 There were no differences in baseline demographic, clinical and laboratory characteristics between the 193 (70.2%) patients with 2 years follow-up and the remaining 82 lost to follow-up. These 193 patients were divided into two groups: 38 (20%) were off therapy (reference group) and 155 (80%) were still on therapy. From these 155 patients we randomly extracted 76 patients and used their baseline data (when the patients were active by inclusion criteria) as a comparison sample in a 1:2 ratio. There were no statistically significant differences in baseline demographic, clinical and laboratory characteristics between the reference and the comparison samples.
Step 1: Clinically inactive disease cut-off selection
Table 1 shows the descriptive statistics related to disease activity/damage characteristics of the 38 patients off therapy included in the reference sample. For those variables where the lowest value correlated with normality, we chose as cut-off the median, 20th percentile or the minimum (eg, 0 for C-HAQ) while, for those measures in which higher values signified inactivity (eg, 52 for CMAS), we selected the median, 80th percentile or the maximum. In addition, we also considered literature-derived cut-off values.
Step 2: Accuracy of inactive disease cut-off values
Table 2 reports the frequency of patients observed with a particular cut-off as well as the results of accuracy measurements for the reference and comparison samples. The frequency of patients fitting a particular cut-off was very high for the reference sample and, conversely, very low for the comparison sample. Exceptions were represented by a low frequency of patients with active JDM but normal serum muscle enzymes, no pain and normal quality of life (CHQ PhS and PsS).
For muscle strength/activity measures, those variables with an almost perfect agreement included MMT=802 or ≥78,7 CMAS ≥48,7 MYOACT ≤0.22 and DAS ≤3 in decreasing order of κ.8 Similarly, MD-GLOVAS ≤0.2 or 0 and physician global assessment of muscle activity (MD-MuscVAS) of 0 had a Cohen's κ >0.8.
Substantial agreement was observed for C-HAQ=0, MYOACT=0, Par GLOVAS (≤ 0.2 or 0), MD-ExtraMuscVAS and for some muscle enzyme levels (normal aldolase, LDH or a combination of normal CPK and LDH). For all the other variables, the cut-off values analysed held an agreement which was slight to moderate. Of note, when the cut-off value was set to the lowest (eg, DAS=0) or maximum values (eg, CMAS=52), the sensitivity and accuracy measurements were lower.
Step 3: Inactive disease criteria testing
Table 3 reports the frequency of patients who fit the top inactive disease criteria tested and their accuracy measurements. The top criteria (C1–C6) required the presence of three of four measures fitting the related cut-off values. They all required the presence of normal muscle enzymes (CPK, aldolase or LDH) as well as muscle strength (MMT) or endurance (CMAS) and a low MD-GLOVAS. Since inactive disease criteria from the literature consistently asked for normal muscle enzymes, the Steering Committee retained them despite their non-optimal performance in Step 2. For example, inactive criterion 1 requires the combination of at least three of four measures being normal according to specific cut-offs: CPK ≤150 or CMAS ≥48 or MMT ≥78 or MD-GLOVAS ≤0.2. Criteria 1, 2, 4 and 6 are similar, with the only difference represented by the muscle enzymes considered (CPK or aldolase or LDH, or the combination of LDH/ALT). Criterion 3 required the use of MYOACT ≤0.2 instead of MD-GLOVAS while criterion 5 required an MMT of 80 instead of ≥78. The 19 criteria derived from the literature consistently showed lower accuracy measures (fair to moderate agreement).
None of the patients in the comparison sample group fitted any of the definitions of inactive disease reported in table 3. Of note, when all criteria were applied to the subgroup of 11 patients who were off therapy at 12 and 24 months (and therefore in clinical remission off therapy for 12 months), their accuracy measures improved with a sensitivity/specificity/κ=1.
For the subsequent analysis, since the top two criteria (C1 and 2) had overlapping accuracy performance with the only difference being that criterion 1 requires normal CPK and criterion 2 requires normal aldolase, it was decided to choose criterion 1 since CPK is more universally used. In this dataset, for example, the frequency of patients with CPK available was 269/275 patients (97.8%) compared with 145/275 patients (52.7%) for aldolase.
Step 4: Inactive disease criteria confirmation
When we applied the top inactive disease criterion 1 (C1) (three of four measures from CPK ≤150, CMAS ≥48, MMT ≥78 or MD-GLOVAS ≤0.2), 30/31 (96.8%) were inactive off therapy at month 24 (reference sample) and 69/145 (47.6%) were inactive on therapy at month 24. Table 4 shows the comparison of disease activity/damage measures in these two groups of patients who met the top inactive disease criterion C1 at month 24. There were no statistically significant differences in the measures examined with the exception of MMT, MD-GLOVAS and CHQ PsS which were slightly worse in the group of patients still on therapy at month 24.
Discussion
Using a data-driven based approach, PRINTO identified a definition to classify JDM patients as clinically inactive on/off therapy if at least three out of four measures meet the proposed inactivity cut-offs: CPK ≤150, CMAS ≥48, MMT ≥78 and PhyGloVAS ≤0.2.
The PRINTO database contains JDM patients with high disease activity at baseline, short disease duration and followed for 24 months.6 ,14 ,45 This time frame was chosen with the rationale that the follow-up period was sufficiently long to induce inactivity. Indeed, after 24 months we identified that 20% of the patients were off therapy. When the disease activity status of this reference sample was analysed, we found that measures related to muscle strength/endurance, disease activity, muscle enzymes and patient's reported outcome had values that were very close to normal. For example, the mean MD-GLOVAS was 0.1±0.2 on a scale of 0–10 cm. The fact that this measure was not exactly equal to 0 can be interpreted either as the physician's aversion to consider an extremely low level of disease activity (close to but not overlapping with 0) in order to discontinue therapies, or as an inherent measurement error of the VAS scale in which a line close to 0 means in reality exactly 0, as also observed in other diseases such as JIA.46 In future studies this measurement error could be avoided with the use of more precise tools such as the 21 circle VAS.47 Similarly, the mean values for muscle strength/endurance were close to but not overlapping with the extreme range of the scales while, for muscle enzymes, all descriptive measures were below the upper range of normal values.
The literature on myositis has numerous definitions of inactive disease which, however, in many cases are not evidence-based or lack the operational cut-off value in order for a particular variable to be called normal.3 ,9 ,38,–,44 In order to overcome these problems, our rationale was to select the best cut-off values for each variable to be able to differentiate active from inactive patients. The variables which showed the best accuracy (Cohen's κ 0.8–1.0) were muscle strength/endurance and some disease activity tools and the MD-GLOVAS, while a substantial agreement (Cohen's κ 0.61–0.8) was observed for parent's reported outcome and muscle enzymes. Of note, most of these measures are part of the PRINTO JDM14 and IMACS48 core set measures for the evaluation of response to therapy. This demonstrates that the core set measures are probably the key measures to evaluate the disease status of patients for each phase of the disease.
One of the challenges in JDM, as well as in other rheumatic diseases, is the lack of a gold standard for the evaluation of the disease status of a patient. By selecting variables for inactivity derived from the literature3 ,9 ,38,–,44 and those based on our Step 2 process, our analytical process allowed us to develop multiple combinations of inactive disease criteria and rank these in order of their best accuracy to characterise an inactive patient. The results showed that all the definitions derived from the literature had only a fair to moderate agreement. The best performing definition of inactive disease, with almost perfect agreement, was related to a definition set up by the Steering Committee. As in the literature, we elected to retain muscle enzymes in the definition despite the fact that their individual accuracy in terms of agreement was substantial but not perfect. The top definitions selected all have some common characteristics, such as a minimum number of individual measures to be observed as normal (in general, three out of four). All the top combinations require the presence of normal muscle enzyme(s), muscle strength/endurance or a MD-GLOVAS close to normal. The similarity of the top definitions and the partial overlap with the variables that are currently used to evaluate response to therapy can be interpreted as a measure of convergent validity of the process.14 ,48 Theoretically, a patient could fit the definition of inactive disease with abnormal muscle enzymes, abnormal muscle strength/endurance or a MD-GLOVAS close to normal. However, it is unlikely that a physician will give a score of ≤0.2 in a child with abnormal muscle enzymes or abnormal muscle strength/endurance unless it was due to damage. It has been reported that patients with longstanding disease and substantial damage cannot achieve normal MMT or CMAS scores,9 but patients in the PRINTO dataset had short disease duration and very minimal damage at baseline.
In the final analytical step we applied the top definition to the remaining part of the sample still on therapy after 24 months of therapy and found that almost 50% of patients fit the definition of inactive disease on therapy, with only minimal clinical and laboratory differences between the group inactive off and on therapy. Similar to other serious paediatric rheumatic diseases, medication reduction can be a challenge in JDM, with physicians hesitant to discontinue medications even when disease is inactive. In comparison to adult myositis, paediatric patients tend to receive corticosteroids and immunosuppressive medications for longer periods with 80% of patients with JDM still receiving medications more than 2 years after the diagnosis despite 50% of them meeting the proposed criteria for inactive disease.6 ,14
The limitations of our study included the fact that the dataset did not have a physician rating of inactive disease, it could not evaluate the predictive ability of the definition to predict subsequent flares, that we were able to identify in this dataset only 11 patients who were in clinical remission off therapy for 12 continuous months, that 30% of the patients were lost to follow-up and that the criteria will need to be validated in a future prospective intervention study.34 ,35 The strength of our criteria lies in the fact that they have been tested in a large number of patients from many countries.
In conclusion, using a data-driven approach, PRINTO established criteria for clinically inactive disease in JDM with evidence-based cut-off values for muscle strength/endurance, muscle enzymes and physical global evaluation of disease activity. These criteria can be used in clinical trials, in research and in clinical practice.
Acknowledgments
The authors thank all members of PRINTO who participated as investigators in the study and whose enthusiastic effort made this work possible.
References
Footnotes
-
Funding Supported by grants from the European Union (contract no. QLG1-CT-2000-00514) and IRCCS G. Gaslini, Genoa, Italy. DL was the recipient of the European League Against Rheumatism (EULAR) Scientific Training Bursaries. PMM attended the PRINTO headquarter at Gaslini Hospital in Genoa (Italy) as part of her visiting professorship sabbatical. LGR was supported by the intramural research program of the National Institute of Environmental Health Sciences, National Institutes of Health.
-
Competing interests None.
-
Ethics approval Ethics approval was obtained from the ethics committees of the participating centres.
-
Provenance and peer review Not commissioned; externally peer reviewed.
-
Data sharing statement The PRINTO database related to this project is open to further research prior to acceptance of the proposal by the PRINTO Advisory Council (www.printo.it).