Disease activity assessment in childhood vasculitis: development and preliminary validation of the Paediatric Vasculitis Activity Score (PVAS)
- Pavla Dolezalova1,
- Fiona E Price-Kuehne2,
- Seza Özen3,
- Susanne M Benseler4,
- David A Cabral5,
- Jordi Anton6,
- Jürgen Brunner7,
- Rolando Cimaz8,
- Katheleen M O'Neil9,
- Carol A Wallace10,
- Nicholas Wilkinson11,
- Despina Eleftheriou2,
- Erkan Demirkaya12,
- Marek Böhm1,
- Petra Krol1,
- Raashid A Luqmani13,
- Paul A Brogan2
- 1Rheumatology Unit, Department of Paediatrics and Adolescent Medicine, Charles University in Prague, 1st Faculty of Medicine and General University Hospital in Prague, Czech Republic
- 2Institute of Child Health and Great Ormond Street Hospital For Children NHS Foundation Trust, UCL, London, UK
- 3Department of Pediatric Rheumatology, Hacettepe University, Ankara, Turkey
- 4Department of Rheumatology, The Hospital for Sick Children, University of Toronto, Toronto, Canada
- 5Department of Pediatric Rheumatology, BC Children's Hospital and University of British Columbia, Vancouver, Canada
- 6Hospital Sant Joan de Déu, Universitat de Barcelona, Barcelona, Spain
- 7Innsbruck Medical University, Innsbruck, Austria
- 8AOU Meyer, Firenze, Italy
- 9University of Oklahoma Health Sciences Center, Oklahoma City, USA
- 10Department of Pediatric Rheumatology, University of Washington and Seattle Children's Hospital, Seattle, USA
- 11Nuffield Orthopaedic Centre NHS Trust, Oxford, UK
- 12Gulhane Military Medical Faculty, School of Medicine, Ankara, Turkey
- 13Department of Rheumatology, NIHR Musculoskeletal Biomedical Research Unit, University of Oxford, Oxford, UK
- Correspondence to Dr Pavla Dolezalova, Rheumatology Unit, Department of Paediatrics and Adolescent Medicine, Charles University in Prague, 1st Faculty of Medicine and General University Hospital in Prague, Ke Karlovu 2, 12800 Praha 2, Czech Republic;
- Accepted 23 September 2012
- Published Online First 25 October 2012
Background Rare chronic childhood vasculitides lack a reliable disease activity assessment tool. With emerging new treatment modalities such a tool has become increasingly essential for both clinical practice and therapeutic trials to reproducibly quantify change in disease state.
Objective To develop and validate a paediatric vasculitis activity assessment tool based on modification of the Birmingham Vasculitis Activity Score (BVASv.3).
Methods A paediatric vasculitis registry was reviewed to identify clinical features missing in the BVASv.3. A modified nominal group technique was used to develop a working version of the Paediatric Vasculitis Activity Score (PVAS). Prospective validation provided tool reliability, reproducibility and responsiveness to change. Training of assessors was done according to the BVAS principles.
Results BVAS items were redefined (n=22) and eight paediatric items added in Cutaneous (n=4), Cardiovascular (n=3) and Abdominal (n=1) sections. The final PVAS has 64 active items in nine categories. The principles of new/worse and persistently active disease were retained as were the overall score and weighting of categories. The median PVAS in 63 children with systemic vasculitis was 4/63 (0–38/63). There was a high interobserver agreement for the overall as well as for subsystem scores (linear-weighted-κ ≥0.87). PVAS correlated with physician's global assessment (p<0.01); treatment decision (p=<0.01) and erythrocyte sedimentation rate (ESR) (p=0.01). In response to treatment, 15/19 patients assessed demonstrated a significant fall in PVAS (p=0.002), with good agreement among assessors for this change.
Conclusions The PVAS validity in children with systemic vasculitis was demonstrated. Like the BVAS, we anticipate that the PVAS will provide a robust tool to objectively define disease activity for clinical trials and future research.
Systemic vasculitides (SV), with the exception of Henoch-Schönlein purpura and Kawasaki disease, are rare in children. Their estimated annual incidence is 0.24/100 000 children.1 Often life-threatening or organ-threatening, these conditions are associated with substantial morbidity and mortality. The evaluation of treatment efficacy is hampered by the absence of a standardised assessment tool with which to define endpoints in multicentre clinical trials to make interstudy comparisons and ultimately to help improve the care of children with vasculitis.
Both the Paediatric Rheumatology European Society (PReS) Vasculitis Working Group and the North American Childhood Arthritis & Rheumatology Research Alliance (CARRA) Vasculitis Committee have together set their long-term aims in vasculitis research to include disease classification and to develop outcome assessment as the priorities. As a result of this endeavour, childhood-specific classification criteria for SV have recently been described and their validation reported.2 ,3 Disease activity has been recognised as a central domain within the core set of outcome measures for clinical trials in systemic vasculitis in adults.4 Designed by consensus of a multispecialty group of vasculitis experts, the Birmingham Vasculitis Activity Score (BVAS) has been validated as a comprehensive multisystem tool for standard assessment of systemic vasculitis and used in multiple clinical trials.5–8 The BVAS V.3 (BVASv.3) has been recommended for use in anti-neutrophil cytoplasmic antibodies (ANCA)-associated vasculitis (AAV) clinical trials as a part of the Outcome Measures in Rheumatology (OMERACT) core set of outcome measures.9
Modified versions of BVAS have been used in observational studies in children.10–12 The performance of BVAS has recently been assessed in patients from north American and European Vasculitis Registries.13 ,14 Despite this preliminary evidence there has remained a need for developing a single, internationally standardised, paediatric-specific tool that would systematically consider the differing clinical manifestations and comorbidities that occur in children 15 while acknowledging the principle that the compatibility with BVAS would allow long-term follow-up of children into adulthood and enable the inclusion of paediatric patients in clinical trials.
In the present study we aimed through international collaborative efforts to develop and validate a paediatric vasculitis activity assessment tool based on modification of BVASv.3 and to establish an effective training system for the use of Paediatric Vasculitis Activity Score (PVAS) according to EULAR recommendations for measuring disease activity.16
BVASv.3 identifies 56 selected features of active vasculitis in nine organ systems distinguishing between new/worse and persistent disease. In order to identify variations and possible additional paediatric items, the clinical data at disease onset from 116 patients entered into the PReS/Pediatric Rheumatology International Trials Organisation (PRINTO) vasculitis registry were reviewed by one of the authors (PD). The disease spectrum covered three main chronic SV: childhood polyarteritis nodosa, granulomatosis with polyangiitis (GPA, formerly Wegener's) and Takayasu arteritis. Items present in at least 20% of the patients in the cohort were included. Subsequent online discussion of items and their definitions among the tool developers (PD, PAB, SÖ, SMB) and the BVAS expert (RAL) resulted in the first working version of a PVAS and its glossary. The new items were assigned weighted scores by consensus while the group weighting of each of the nine organ categories remained unchanged from the BVASv.3.
Twenty paediatric paper cases were prepared (PD, RAL) covering all PVAS items of active vasculitis mixed with features attributable to disease damage or comorbidities, using principles of the BVAS training cases. A working group of 11 paediatric rheumatologists with expertise in vasculitis was established from members of PReS and CARRA. At the first consensus meeting the principles of vasculitis activity assessment and PVAS use were reviewed. All investigators subsequently completed the 10 beginner-level and 10 advanced training paper cases each followed by feedback with ‘correct’ scoring and online discussion of potential PVAS inaccuracies. A second consensus meeting resulted in minor modifications and established face and content validity of the tool.
To further validate PVAS, disease activity was prospectively assessed in children with SV in three centres: Great Ormond Street Hospital NHS Foundation Trust, London; Hacettepe University Hospital, Ankara; and General University Hospital in Prague. The study was approved by local ethics committees and informed consent was obtained from parents/guardians of all participants and participants aged 16–18 years. The inclusion criteria were as follows: patients aged 0–18 years seen in clinics and hospital wards with systemic vasculitis of any type and duration confirmed by histopathological and/or arteriographical assessment. Anonymised clinical data were recorded on a standardised form and included diagnosis, demographics, therapies from disease onset, treatment decision, physician's global assessment (PGA, 100 mm visual analogue scale), C-reactive protein (CRP) and erythrocyte sedimentation rate (ESR). Treatment decision was defined in six categories according to Suppiah et al.17 Correlation of PVAS with other measures of disease activity was performed in order to assess convergent validity. In order to test the reproducibility of PVAS, an additional five clinicians from the three centres underwent training in PVAS use according to the manual by Luqmani et al.18 PVAS training score sheets were evaluated by a single investigator (PD) for agreement with predefined ‘master scores’. The pass mark was set at an average of 85% agreement overall, with no individual case mark falling below 50% agreement with the master score. Agreement was defined as ‘complete agreement with the master score’ for each individual section of PVAS. Disagreement was defined as ‘any number of items varying from the master scores for each individual section’. This included appropriate use of the ‘none’ box when no active vasculitis items were present within the category.
Patients underwent independent PVAS assessment by two observers on the same day. During routine follow-up patients were assessed a second time to examine tool responsiveness to change in disease state.
Sample size calculations were based on the correlation between the PVAS and the PGA. In two studies of the BVAS and a study of the Vasculitis Activity Index (another adult tool for scoring vasculitis disease activity), correlation with an ‘informed’ PGA was 0.91, 0.84 and 0.85 respectively.5 ,17 ,19 We recognised that PGA scoring might not have been independent of BVAS assessment because the PGA was completed only after the scoring physician completed the BVAS and this process in itself would inform the physician's concept of disease activity and its global assessment for that individual patient. We anticipated that a correlation of at least 0.80 would thus represent a clinically relevant correlation between the PVAS and PGA. To detect a correlation of 0.80, with a two-sided significance level set at 5% and a power of 0.9, a minimal sample-size of 12 would be required. For clinical validity, however, we wished to include a minimal sample size of 50 subjects. All data analyses were conducted using MedCalc Software V.22.214.171.124 and SPSS V.19. Non-parametric statistical tests were used where the data groups could not be uniformly transformed to normality. Interobserver variability was assessed using Bland-Altman limits of agreement for the PVAS overall-scores and linear-weighted κ statistic (Κ) for organ-system subscores. Correlations with known markers of disease activity were measured using Spearman's rank method. Where the PVAS differed between the two assessors, the mean PVAS was used in the calculation. In instances where more than one observation were available in a single patient, measurements from the patient's first visit were used for correlation. Tool-responsiveness to change was analysed using a Wilcoxon-signed rank test to compare repeated measures over time.
Twenty-two original BVASv.3 items were redefined (see online supplemental table S1) and eight new items were added to the three PVAS sections (table 1).
PVAS incorporates age-specific reference ranges for blood pressure and renal function and a paediatric definition of weight loss. The final PVAS is a clinical index of 64 manifestations of active vasculitis. As in BVAS, each item is allocated to one of nine organ-based systems. The scoring sheet simply records the presence or absence of each item, which must be attributable to active vasculitis after exclusion of other causes such as infection. Active disease is defined as the presence of a feature that is new or worse within the last 4 weeks; items that have been present for more than 4 weeks but less than 3 months are considered to be persistent. Every item has an assigned score in the ‘new/worse’ and ‘persistent’ scale, weighted according to severity. The BVASv.3 ‘persistent disease only’ concept has been retained in PVAS with the minor modification towards ‘no new/worse disease’. As in BVASv.3, the new/worse scale has a maximum score of 63, while the persistent scale has a maximum overall score of 33. The final PVAS and accompanying glossary of definitions for each item are available as online supplementary table S1 and S2.
All investigators underwent a uniform training procedure. Based on an ideal answer for each of the 20 paper cases 9/11 individuals involved in the tool development reached 85% or higher agreement with master scores. The main source of disagreement was the failure to record the absence of any items within a section (the ‘None’ box). All investigators who contributed to PVAS prospective validation reached the pass mark.
Prospective PVAS validation
Sixty-three children (35 girls, median age 12 years) with new or existing diagnoses of SV were recruited (table 2).
The median disease duration at initial assessment was 12.0 months (0–194). There were 38 patients (60.3%) with active disease and 25 patients (39.7%) in remission (PVAS=0). The overall median PVAS was 4/63 (range 0–38/63). Table 3 shows the distribution of organ involvement according to PVAS subcategory at the time of initial assessment.
At study inclusion 29 patients (46.0%) were taking oral prednisolone and eight patients (12.7%) were being treated with high dose intravenous methylprednisolone. One patient with severe life-threatening GPA (PVAS=38/63) was undergoing plasma exchange (combined with high dose corticosteroids) and nine patients (14.3%) were receiving IV cyclophosphamide. Ten patients were taking azathioprine (15.9%), 10 mycophenolate mofetil (15.9%) and 19 patients (30.2%) were taking other disease-modifying antirheumatic drugs: methotrexate (n=5); anti-tumour necrosis factor (TNF) agents (n=4); colchicine (n=10) and rituximab (n=1).
In the 55 patients assessed by two independent observers on the same day, agreement between the assessors’ overall PVAS scores was high; the assessors agreed on the overall score for 45/55 patients (82%). The mean difference (d¯) of the assessors’ PVAS scores was zero with 95% limits of agreement of −1.85 and 1.85, demonstrating that for a new patient, the two assessors would differ by less than two PVAS-points, with the discrepancy being equally likely in each direction.20 The Bland-Altman plot (figure 1) demonstrates that the numeric value of the PVAS did not influence the agreement between the assessors with comparable high levels of agreement for low, moderate and severe disease activity.
A linear-weighted κ (Κ) statistic for the organ-system scores showed high agreement between assessors for all sections, with the lowest K score of 0.87 (95% CI 0.71 to 1.00), in the ‘ear, nose and throat’ section (table 3); the perfect agreement we observed between assessors (Κ=1) for the ‘chest’ and ‘cardiovascular’ subscores may reflect a very low prevalence of these organ manifestations. The ‘other’ section was used to record miscellaneous items using free text (including vestibular dysfunction; testicular swelling; testicular haematoma and necrosis; paraesthesia; clinical pancreatitis; pseudofolliculitis). Overall agreement between assessors for this section (including no item listed) was 93% (n=51/55). In 14/16 of these patients where interobserver analysis was available agreement for items listed was 71% (n=10/14).
A strongly positive correlation was demonstrated between the PVAS and PGA (Spearman's rank correlation (rs)=0.87, 95% CI 0.79 to 0.92, p<0.01). When patients in remission (PVAS=0) were excluded from the analysis (n=25), correlation remained high (rs=0.60, 95% CI 0.33 to 0.78, p=0.0001).
ESR and CRP at the initial assessment were available in 46 and 48 patients respectively. There was a positive correlation between the PVAS and ESR (rs=0.37, 95% CI 0.09 to 0.6, p=0.01). Again, when patients in remission were excluded from the analysis, the correlation remained significant (rs=0.39, 95% CI 0.05 to 0.64, p=0.01). There was no significant correlation between the PVAS and CRP levels (rs=0.21, 95% CI −0.08 to 0.46, p=0.16) and when patients in remission were excluded, the correlation remained non-significant (rs=0.17, 95% CI 0.17 to 0.47, p=0.32).
In all 63 patients treatment decision at the time of the assessment was available. Figure 2 documents correlation between PVAS and treatment decision categories (rs=0.47, 95% CI 0.17 to 0.68, p=<0.01) as defined in the recent BVASv.3 validation excercise.17
Nineteen patients were assessed on more than one occasion to examine the response of PVAS to change in clinical status. The median follow-up time between assessments was 4 (1–77) months. Fifteen patients demonstrated a reduction in PVAS score in response to treatment: 66.6% patients (n=10/15) had treatment decision category six (‘major escalation’) at the first assessment. The median PVAS at first and follow-up assessments was 9/63 (4–38/63) and 2/63 (0–16/63), respectively (p=0.002) (figure 3). There was a corresponding reduction in the treatment decision category numbering for 14/15 patients (data not shown).
Three patients with stable disease (PVAS unchanged) had treatment decision category 3 at the first assessment. Two of them (PVAS=0) were able to either stop treatment (category 1) or continue minor treatment (category 3) at second assessment. One patient with ongoing disease activity (PVAS=6) required treatment decision category 4 (‘minor escalation of treatment’) at second assessment. One patient suffered a minor relapse of GPA affecting the upper respiratory tract: the PVAS increased from zero at the first assessment to 5/63 at follow-up.
Physicians need to measure disease activity in order to make therapeutic decisions, list disease manifestations, evaluate prognosis, compare patient groups and assess treatment response over time.21 In the absence of a single reliable biomarker of vasculitis activity, a structured comprehensive assessment appears as a satisfactory solution. We have developed a paediatric-appropriate modification of BVAS and demonstrated clinical utility and validity of a newly created tool according to the OMERACT filter for outcome measures in rheumatology.22 PVAS appears to be a feasible tool with face, content and convergent validity and is reproducible and sensitive to change in patient clinical status. The design of PVAS has retained the principles of the current adult vasculitis activity assessment including the concept of new/worse and persistent disease. Redefinition of items in the Glossary agreed by a multilingual investigator group reflects a paediatric clinical approach and internationally understandable English explanation of features that are rarely seen in the daily practice of most paediatricians. The addition of new disease items based on the review of the PReS/PRINTO vasculitis database has further improved paediatric-specific content validity of the tool. The PVAS feasibility was not expected to differ from that confirmed for BVAS through its use in various versions in numerous clinical trials.23 ,24 Accordingly, completion of the PVAS form takes less than 3 min when performed by a trained clinician.
PVAS reliability has been demonstrated by its low interobserver variability, with consistent PVAS values obtained by independent assessors for both the overall scores, and organ-system subscores. This excellent interobserver agreement is similar to that obtained in the validation of the BVASv.3, and also for the BVAS/WG.5 ,17 ,25
PGA of disease activity has been successfully used in other paediatric conditions, including Juvenile Idiopathic Arthritis core set of outcome variables.26 Some controversy exists regarding the usefulness of the PGA in adults with vasculitis, some arguing that it is completely meaningless unless linked to physician intention to change treatment.7 A retrospective study using the PReS/PRINTO vasculitis database2 ,3 attempted to analyse validity of BVAS for assessing disease activity in children.14 This study reported a poor correlation of BVAS with the PGA, supporting the concept that the PGA is influenced significantly by factors other than disease activity, and is in itself, by its very nature, very subjective. More recently, Morishita et al in a retrospective study of a registry of children with ANCA-associated vasculitis observed a weak positive correlation of BVASv.3 with PGA.3 The authors concluded that prospective evaluation of BVASv.3 and/or a paediatric-specific modified tool was now required. In contrast, we demonstrated excellent correlation between the PVAS and the informed PGA, (ie, performed after completion of PVAS), replicating the strong association shown in adults, but also only if completed after BVAS had been evaluated.5 ,17 The persisting strong correlation when patients in remission were excluded reduces the possibility that this correlation was artificially raised by physicians assigning zero value to both the PVAS and the PGA. Important reasons explaining why PVAS correlated better with PGA than BVASv.3 are, first, PVAS was scored prospectively in real-time by clinicians formally trained to use the tool; second, prospective PVAS scoring and PGA were obtained from the same assessor; and third items within the PVAS have been altered to be more relevant to childhood vasculitis.
We did not observe a significant correlation between the PVAS and CRP. While Hind et al27 ,28 demonstrated a good correlation between CRP and overall disease activity assessment, more recent studies have not supported this.5 ,18 Jayne et al29 found that changes in CRP may correspond to changes in disease activity on a population level but not necessarily in individual patients. There are several potential explanations. First, PVAS items are scored as active if present in the preceding 4 weeks, even if in the interim they have improved significantly, for example, due to escalation in treatment. It is therefore possible that these ‘active items’ could have already responded to treatment, with corresponding fall in CRP but still score positively on the PVAS. Alternatively, the CRP may have responded to the therapy even though the clinical signs have not yet subsided.7 Lastly, some vasculitic features (eg, minor cutaneous flare) may not necessarily result in high CRP. We did demonstrate a positive correlation between the PVAS and ESR. ESR may be slower to react to changes in systemic inflammation than CRP, and therefore may more closely reflect the PVAS since both are influenced by clinical events in the preceding weeks. Arguably ESR reflects disease activity less specifically than CRP as it can be affected by haemoglobin level and other factors, such as chronic kidney disease or dysfibrinogenaemia, not directly related to vasculitis activity.30–32
The limited amount of longitudinal data is one of the study weaknesses. This is a consequence of the rarity of systemic vasculitides in childhood. Responsiveness of BVASv.3 was assessed in 39 patients after ‘major escalation’ of treatment.5 From 19 children where follow-up data were available in our study only 10 had such a treatment change.
Training in the application of the vasculitis activity concept and in the practical use of the tool ensures optimum reliability and reproducibility of BVAS and its adaptations.5 ,17 Considering the rarity of vasculitis in the paediatric population and consequent relative lack of expertise in this area, such training is vitally important. In this study, all investigators accomplished the scheduled training and requisite pass-mark, and we emphasise the importance of this point particularly if the tool is to be used in future multicentre clinical trials.
The relative merits of a generic disease activity measurement tool such as BVAS (vs a disease specific tool) for use in evaluating patients with large vessel vasculitis has been questioned.33 In this study the disease activity tool was assessed in a limited number of children with large vessel vasculitis and therefore our findings may also not be readily generalisable beyond patients with small to medium vessel vasculitis.
This is the first attempt to validate a new disease assessment tool for paediatric vasculitis. Similar to adult instruments, PVAS should remain open to revisions including data-driven items and weighting changes. The rarity of multisystem vasculitis has, in part, limited the assessment of the practical application of PVAS to a relatively small number of patients. We plan further validation of the tool in an independent data set to assess its wider applicability. As with the BVAS, we anticipate that the PVAS will robustly and objectively define disease activity as one of the domains of the future core set of vasculitis outcome measures for use in clinical trials and research involving children with vasculitis.
The study consensus meetings were supported by Paediatric Rheumatology European Society (PReS) and Childhood Arthritis & Rheumatology Research Alliance (CARRA). The principal investigator was supported by the Charles University Grant Agency GAUK 52608/2008.
Handling editor Tore K Kvien
Contributors Principal investigator and study co-ordinator: PD Joint senior authors: RAL, PAB Paediatric Vasculitis Activity Score (PVAS) developers: PD, RAL, PAB, SMB, SÖ Nominal group technique and consensus conference participants: PD, SÖ, SMB, DAC, JA, JB, RC, KMO, CAW, NW, PAB, RAL Prospective validation of the PVAS: FEP-K, PAB, PD, DE, SÖ, ED, MB, PK Statistical analysis: FEP-K.
Competing interests None.
Funding Consensus meeting support only.
Patient consent Obtained.
Ethics approval Ethical Committees or IRB of institutions involved in the validation process.
Provenance and peer review Not commissioned; externally peer reviewed.