The need for revision

Most recent clinical trials in vasculitis have used a BVAS1 of zero as a definition of remission. (reviewed by Mukhtyar et al).19 Ignoring BVAS2 (persistent disease) underestimates true disease activity. This has therapeutic implications because persistent disease needs a different approach.16 Redundant and/or uncommon items have been identified in BVAS(v.2).20 Removal of those items would increase the feasibility of the modified clinical tool.

Aim

To validate a new version of BVAS for use in studies of systemic vasculitis.

Methods

The new clinical tool (see supplementary online appendix) was designed by consensus (face validity).20 The number of items was reduced from 66 in BVAS(v.2) to 56 in BVAS(v.3) either by omission or by merging with other items. The “persistent” boxes for each item were replaced by a single “persistent” box for the whole form. This box was marked, only if every disease manifestation was attributable to “persistent” disease. All items were treated as “new/worse” if any of them were “new/worse”. Disease manifestations were recorded if they had been active in the previous 4 weeks, and were directly attributable to vasculitis.

Weighting was largely unchanged, but new items were given weights by consensus. An extensive glossary and manual of operations was constructed. The BVAS(v.3) forms, training manual and glossary sheet are available online at the EUVAS website (http://www.vasculitis.org/disease.htm, accessed 13 July 2009).

Nineteen subjects completed 20 beginner-level training paper cases, and 14 of them, including all investigators recruiting patients for this study, completed 40 advanced-level paper cases. This paper-case exercise had two benefits; it trained each investigator in the use of BVAS(v.3) and provided information on interobserver reliability.

Inclusion criteria: Patients seen in outpatient clinics and hospital wards with a probable or definite diagnosis of systemic vasculitis, of any duration, at any stage of their disease, were recruited from nine UK centres: Birmingham (Birmingham City Hospital), Cambridge (Addenbrooke’s Hospital), Edinburgh (Western General Hospital), Norfolk (Norfolk and Norwich University Hospital), Nottingham (Nottingham University Hospital), Oxford (Oxford Radcliffe Hospitals and Nuffield Orthopaedic Centre), Reading (Royal Berkshire Hospital) and Westcliff-on-sea (Southend General Hospital). The investigators were rheumatologists (CM, DC, SD, PL, LY, RAL), nephrologists (OF, DJ, RJ), or allied health professionals with an interest in vasculitis (DB, CH, JH, AM).

Exclusion criteria: Patients with a probable or definite diagnosis of giant cell arteritis were excluded.

The study was approved by local ethics committees and all patients gave written informed consent. The data recorded on a standardised case record form included demographics, diagnosis, treatment in the previous 5 years, serum C-reactive protein (CRP), serum creatinine, antineutrophil cytoplasmic antibody (ANCA) status with titres and antibody levels, BVAS(v.3), treatment decision, physician’s global assessment (100 mm visual analogue scale), vasculitis activity index (five-point Likert scale, with 0 signifying remission and 4 suggesting maximum activity). Patients in Oxford and Edinburgh were also assessed using BVAS(v.2) at the same visit. Table 1 shows the treatment decisions classified into six ordered categories. When treatment decisions did not match the definitions, the category was jointly assigned by RAL and CM. The BVAS values for both versions were calculated on a Microsoft Excel based program developed by CM.

Convergent validity was assessed by examining the association of BVAS(v.3) with

• BVAS1 subscore of BVAS(v.2) in patients who were assessed using both clinical instruments at the same hospital visit;

• treatment decision (table 1);

• nearest serum CRP level within 1 month of the consultation (when values were reported to be less than the lowest measurable value for that laboratory, the value was recorded as 0 mg/l for the purpose of statistical analysis);

• physician’s global assessment;

• vasculitis activity index.

The ability of the BVAS(v.3) to record persistent disease was assessed in patients with persistent disease on a follow-up assessment with BVAS(v.2). They were defined as having persistent disease if the BVAS2 subscore was higher than the BVAS 1 subscore.

Interobserver reliability (reproducibility) of BVAS(v.3) was examined in patients independently assessed by two observers on the same day. This was also assessed in the paper-case exercise.

Intraobserver reliability (repeatability) of BVAS(v.3) was examined in the subgroup of patients with a second assessment by the same observer within 9 days of the first assessment.

Sensitivity of BVAS to a change in disease state was assessed in a separate cohort of patients with a new diagnosis of ANCA-associated vasculitis. Thirty-nine patients were assessed at diagnosis and at 3 months using the BVAS(v.3), following treatment which was categorised as “major escalation” (table 1). It was expected that the patients would improve with the treatment and the change in BVAS was assessed as a marker of sensitivity to change in disease status.

Statistical analysis

The Spearman rank correlation coefficient was used to assess the association of BVAS(v.3) with subscores of BVAS(v.2), treatment decision, serum CRP levels, physician’s global assessment and vasculitis activity index. Reliability of the BVAS(v.3) was assessed using the intraclass correlation coefficient (ICC), and the linear weighted κ statistic for the organ-system subscores. Sensitivity to change was assessed by paired t test comparing BVAS(v.3) scores at 0 and 3 months. The statistical analysis was performed using SPSS 15.0 except for the linear weighted κ analysis which was performed using an online statistics package (http://faculty.vassar.edu/lowry/VassarStats.html, accessed 13 July 2009).

Results

Three hundred and thirteen patients with systemic vasculitis were recruited between 28 July 2004 and 11 December 2007. The mean (SD) age of the cohort was 55.0 (15.9); range 18–87 years. There were 162 women (52%), and 149 men (48%). Table 2 shows the demographics of the cohort. The median disease duration was 3.8 years (lower and upper quartiles 1.0, 8.7 years). The median BVAS(v.3) was 2 (for histogram see supplementary online figure 1).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Correlation between treatment decision and the Birmingham Vasculitis Activity Score (version 3) (BVAS(v.3)).

Convergent validity

Of 313 patients, 302 had a recorded treatment decision. There was a correlation between the treatment decision and the BVAS(v.3) (Spearman’s correlation coefficient (r_s) = 0.66, 95% CI 0.59 to 0.72) (fig 1). Subgroup analysis in 153 patients with Wegener’s granulomatosis revealed a similar correlation (r_s = 0.72, 95% CI 0.64 to 0.79) (see supplementary online fig 2).

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Correlation between the Birmingham Vasculitis Activity Score (version 3) (BVAS(v.3)) and the BVAS1 subscore.

One hundred and thirty-eight patients were assessed using the BVAS(v.2) and the BVAS(v.3). The correlation between the BVAS(v.3) and the BVAS1 subscore was strong (r_s = 0.94, 95% CI 0.92 to 0.96) (fig 2). On follow-up, 19 patients were found to have persistent disease. Assessment with BVAS(v.2) and BVAS(v.3) at the same consultation demonstrated a strong correlation between BVAS(v.3) and the BVAS2 subscore of BVAS(v.2) (r_s = 0.60, 95% CI 0.21 to 0.83).

One hundred and eighty-five patients had CRP levels available within 4 weeks of their BVAS assessment. There was a moderate correlation between BVAS(v.3) and CRP levels (r_s = 0.43, 95% CI 0.31 to 0.54) (see supplementary online fig 3).

• Download figure
• Open in new tab
• Download powerpoint

Figure 3

Birmingham Vasculitis Activity Score (version 3) (BVAS(v.3)) at diagnosis and at 3 months.

A strong correlation of BVAS(v.3) was demonstrated with a physician’s global assessment available for 307 patients (r_s = 0.91, 95% CI 0.89 to 0.93) (see supplementary online figure 4), and a five-point Likert scale vasculitis activity index available for 304 patients (r_s = 0.88, 95% CI 0.86 to 0.91) (see supplementary online fig 5).

Reliability

Reproducibility of BVAS(v.3) for the 20 basic-level paper cases completed by 19 observers was high with an ICC of 0.89 (95% CI 0.81 to 0.94). In the 40 advanced level paper cases completed by 14 observers the reproducibility was high with an ICC of 0.95 (95% CI 0.93 to 0.97).

In the 99 patients independently assessed by two observers on the same day the reproducibility of BVAS(v.3) was high with an ICC of 0.96 (95% CI 0.95 to 0.97). The reproducibility as measured by the linear weighted κ statistic was high for each of the nine organ-system subscores with the lowest weighted kappa being 0.78 (κ scores available as online supplementary table 1).

Repeatability of BVAS(v.3) in 39 patients assessed twice within 9 days, by the same investigator was high with an ICC of 0.96 (95% CI 0.92 to 0.97). A linear weighted κ statistic for the organ-system subscores showed good repeatability, minimum linear weighted κ was 0.75 (κ scores available as online supplementary table 1).

Sensitivity to change in disease status

Thirty-nine patients with a new diagnosis of ANCA-associated vasculitis (11 microscopic polyangiitis, 8 renal limited vasculitis, 20 Wegener’s granulomatosis) were assessed at diagnosis and 3 months using the BVAS(v.3). Each patient received treatment classified as “major escalation” (table 1). At diagnosis the mean (SD) BVAS(v.3) was 18.92 (6.06). Three months after treatment, the mean (SD) BVAS(v.3) fell to 2.03 (2.48) (fig 3). The mean fall in BVAS(v.3) of 16.9 (95% CI 14.8 to 18.9) was statistically significant (p<0.001 using paired t test). During this period the mean fall in the CRP levels was 40.0 (95% CI 15.4 to 64.7) mg/l.

Discussion

We have shown in this study that BVAS(v.3) has face validity, convergent validity with a number of parameters, and it is repeatable, reproducible and sensitive to changing disease states. In comparison with the previous validation studies of BVAS, this study is more robust, having larger patient numbers, wider breadth of vasculitis syndromes and a comparison with a higher number of parameters (table 3). Like previous validation exercises, we have not included assessment of giant cell arteritis. Giant cell arteritis is a distinct clinical syndrome with a limited rage of abnormalities as measured using BVAS items. This homogeneity of clinical manifestations would produce a limited range of scores and is not conducive to activity assessment using BVAS.

The BVAS(v.3) was designed by consensus of a multispecialty group of vasculitis experts. It is a list of 56 items considered to be vasculitis manifestations with a numerical weight attached to each item, and each organ system has a ceiling score. These scores reflect the proportional importance of each manifestation and each organ system. This design of the BVAS(v.3) satisfies the conditions to meet “face validity”.

An ideal clinical tool should possess the ability to differentiate between diseases and/or disease states of interest. This ability of BVAS(v.3) can potentially be assessed in three different ways: first, the ability to differentiate between vasculitis and other conditions; second, the ability to differentiate active disease from damage; and lastly, the ability to differentiate between active disease, persistent disease and remission. The first two abilities are difficult to test owing to the rules governing the scoring of the BVAS(v.3). The marking of any manifestation on the BVAS sheet requires the doctor to be able to attribute its presence to active vasculitis. Strict application of this rule would automatically give all patients without vasculitis a score of zero, creating a spurious difference. In 39 patients we assessed the sensitivity of BVAS(v.3) to a change in disease status. In the absence of a valid external comparator, it is difficult to interpret a change in BVAS, but a fall of over 16 units is clinically meaningful. This improvement in clinical assessment was associated with a fall in CRP levels. There is no relationship between the magnitudes of change for the BVAS(v.3) and the CRP levels.

The BVAS(v.3) correlates only moderately with the CRP. This suggests that the inflammatory response is of value in making treatment decisions, but only in the correct clinical context. For example, the subject with the highest BVAS in our study (BVAS = 37) had active disease in five organ systems but had a CRP of <10 mg/l. Potential causes for this moderate correlation include—generation of CRP due to causes other than vasculitis, and treatment in the preceding 4 weeks producing a spuriously low CRP at the time of recruitment. A previous study did not find any correlation between CRP and disease activity.21 This study demonstrates an excellent correlation between the physician’s global assessment and the BVAS(v.3) score. This result is consistent with the results of Stone 2001,11 but not with those of Luqmani 1994 (table 3).9 This discrepancy can be explained by a fundamental difference in the way in which the exercise was carried out in the three studies. In Luqmani 1994, the physician’s global assessment and the BVAS assessment were independently done by two investigators. In this study, as in Stone 2001, the physician’s global assessment was performed by the same investigator after the BVAS assessment. It can be concluded that a physician’s global assessment in itself may not correlate significantly with disease activity, but a physician’s global assessment informed by the completion of a structured clinical interview and examination would have a good correlation with disease activity.

The BVAS(v.3) correlates with the treatment decision, but this correlation is not absolute. Of 67 patients who had a “major escalation” of their treatment, 12 had a BVAS(v.3) <5. This may be due to two reasons. First, initial treatment with prednisolone may have reduced disease activity. If the patient was recruited more than 4 weeks after the onset of symptoms, disease activity would be low but the start of an immunosuppressive maintenance regimen would be classified as “major escalation”, resulting in a discrepancy. Second, a rise of a few units may be a major flare in patients with certain vasculitis syndromes needing major escalation of treatment.

The BVAS(v.2) has been the “gold standard” for assessment of disease activity in clinical trials of systemic vasculitis. The excellent correlation between the two clinical tools makes the BVAS(v.3) a better tool owing to its ability to record persistent disease activity without the need for a separate subscore and its relative ease of use. It does not take any more time to complete than a structured clinical assessment. We have designed a Microsoft Excel based BVAS calculator which offers free online scoring of the BVAS at our website http://www.ndos.ox.ac.uk/research/rluqmani (accessed 13 July 2009). The calculator is a modification of the Microsoft excel programme developed for this study.

The BVAS(v.3), in common with all structured clinical assessments in vasculitis requires initial training, for which we have made the training manual freely available online at http://www.vasculitis.org/disease.htm. Training ensures that clinical observers, irrespective of their specialty, agree on what to score as directly attributable to active vasculitis.

The BVAS(v.3) is validated and ready for use in clinical trials. The focus of the BVAS has always been on clinical trials, but it may be of value in daily clinical practice. There is evidence that relapse of disease activity can occur in a previously unaffected organ system, so routine clinical examination and interview should be comprehensive and structured.22 The BVAS(v.3) serves as a checklist of items to screen for in daily practice. In the past, the complex scoring system has deterred routine clinical use of the BVAS, but the online BVAS calculator resolves that problem. With adequate prior training in the use of BVAS(v.3), the specialist and the general doctor can use the online calculator to convert disease activity into a tangible score giving them an accurate idea of the disease activity of their patient.

Disease-specific clinical assessment tools for vasculitis have been developed for Wegener’s granulomatosis (BVAS/WG) and Takayasu arteritis (Indian Takayasu Arteritis Score).11 23 These clinical tools may be more specific than BVAS when used for assessing disease activity in those conditions, although this has not been formally tested. However, early disease may not be accurately classifiable or the vasculitis syndrome may have overlap features.24 25 In these situations, the BVAS(v.3) may be used in preference to the disease-specific clinical assessment tools. The magnitude of activity as suggested by the BVAS(v.3) is not comparable across well-differentiated vasculitis syndromes, especially when they do not involve blood vessels of similar calibre, consistent with previously reported data.9 However, disease-specific modules could be developed to supplement BVAS, especially for diseases with a limited number of systems involved such as Takayasu arteritis. By retaining a generic set of items, it will be possible to compare core data between different vasculitis syndromes in order to provide more information on the long-term status of patients with systemic vasculitis, who now have a much improved life expectancy as a result of modern treatment. There may be a relationship between BVAS(v.3) and mortality but this could not be examined in our cross-sectional study.

Inevitably, with more extensive use, this version of BVAS may require editing and revision, as would be expected in any biological measurement.. However, BVAS (v3) currently represents a robust and useful tool for standard assessment of systemic vasculitis.