Article Text

Extended report
Modification and validation of the Birmingham Vasculitis Activity Score (version 3)
  1. C Mukhtyar1,
  2. R Lee2,
  3. D Brown1,
  4. D Carruthers3,
  5. B Dasgupta4,
  6. S Dubey5,
  7. O Flossmann6,
  8. C Hall2,
  9. J Hollywood4,
  10. D Jayne6,
  11. R Jones6,
  12. P Lanyon7,
  13. A Muir7,
  14. D Scott5,
  15. L Young8,
  16. R A Luqmani1,2
  1. 1
    University of Oxford, Oxford, UK
  2. 2
    University of Edinburgh, Edinburgh, UK
  3. 3
    Birmingham City Hospital, Birmingham, UK
  4. 4
    Southend University Hospital NHS Foundation Trust, Westcliff-on-Sea, UK
  5. 5
    Norfolk and Norwich University Hospital Trust, Norwich, UK
  6. 6
    Addenbrooke’s Hospital, Cambridge, UK
  7. 7
    Nottingham University Hospitals NHS Trust, Nottingham, UK
  8. 8
    Royal Berkshire NHS Foundation Trust, Reading, UK
  1. Correspondence to Dr R A Luqmani, Botnar Research Centre, University of Oxford, Windmill Road, Oxford OX3 7LD, UK; raashid.luqmani{at}ndos.ox.ac.uk

Abstract

Background: Comprehensive multisystem clinical assessment using the Birmingham Vasculitis Activity score (BVAS) is widely used in therapeutic studies of systemic vasculitis. Extensive use suggested a need to revise the instrument. The previous version of BVAS has been revised, according to usage and reviewed by an expert committee.

Objective: To modify and validate version 3 of the BVAS in patients with systemic vasculitis.

Methods: The new version of BVAS was tested in a prospective cross-sectional study of patients with vasculitis.

Results: The number of items was reduced from 66 to 56. The subscores for new/worse disease and persistent disease were unified. In 313 patients with systemic vasculitis, BVAS(v.3) correlated with treatment decision (Spearman’s rs = 0.66, 95% CI 0.59 to 0.72), BVAS1 of version 2 (rs = 0.94, 95% CI 0.92 to 0.96), BVAS2 of version 2 in patients with persistent disease (rs = 0.60, 95% CI 0.21 to 0.83), C-reactive protein levels (rs = 0.43, 95% CI 0.31 to 0.54), physician’s global assessment (rs = 0.91, 95% CI 0.89 to 0.93) and vasculitis activity index (rs = 0.88, 95% CI 0.86 to 0.91). The intraclass correlation coefficients for reproducibility and repeatability were 0.96 (95% CI 0.95 to 0.97) and 0.96 (95% CI 0.92 to 0.97), respectively. In 39 patients assessed at diagnosis and again at 3 months, the BVAS(v.3) fell by 17 (95% CI 15 to 19) units (p<0.001, paired t test).

Conclusion: BVAS(v.3) demonstrates convergence with BVAS(v.2), treatment decision, physician global assessment of disease activity, vasculitis activity index and C-reactive protein. It is repeatable, reproducible and sensitive to change. The new version of BVAS is validated for assessment of systemic vasculitis.

View Full Text

Statistics from Altmetric.com

The systemic vasculitides are heterogeneous conditions resulting in inflammation of blood vessels. They can be primary, or secondary to other autoimmune conditions, drugs or infections.1 2 3 Depending on the size of the blood vessels involved and the organ distribution of the vasculitis, the primary systemic vasculitides are classified into discrete clinical syndromes.4 Systemic vasculitis commonly involves multiple organ systems and can present to any specialty. There are no absolute tests or clinical criteria for diagnosing vasculitis. The biopsy yield depends on the sampled organ and varies considerably.5 6 7 Pattern recognition and a standardised assessment technique assists diagnosis and monitoring of these conditions (reviewed by Nataraja et al).8

The Birmingham Vasculitis Activity Score (BVAS) was validated for assessment of disease activity in systemic vasculitis in 1994.9 It was modified in 1997—BVAS(v.2)—for use in collaborative European trials; and again in 2001 to produce a disease-specific instrument for Wegener’s granulomatosis (BVAS/WG).10 11 BVAS(v.2) has been used in clinical trials for assessment of disease activity, defining entry criteria, defining remission and relapse and as an outcome measure.12 13 14 15 16 The European League Against Rheumatism (EULAR) recommendations for the conduct of clinical trials in systemic vasculitis advocate the use of the BVAS to standardise disease assessment in clinical trials.17 BVAS(v.2) may have prognostic value; patients with a high BVAS(v.2) at diagnosis may have a greater risk of mortality.9 BVAS(v.2) has been shown to have a strong correlation with the five factor score (Spearman correlation coefficient (rs) 0.69).18

BVAS(v.2) is a list of 66 manifestations of systemic vasculitis, divided into nine organ-based systems. Each item can either be “new or worse” if it has presented, recurred or worsened within the previous 4 weeks, or “persistent” if it is present but not worse than in the previous 4 weeks. The manifestations have to be attributable to active vasculitis because they may be due to sequelae of previous activity, drug induced, or due to comorbidities. Lack of attribution may cause spurious elevation of the BVAS. Each item has a predetermined numerical score, which may be different depending on whether an item is new/worse or persistent. Each organ system has a predetermined ceiling score. The sum of the scores of all the organ systems reflects disease activity. The BVAS(v.2) generates two scores; BVAS1 reflects new or worse disease, and BVAS2 reflects persistent disease.

The need for revision

Most recent clinical trials in vasculitis have used a BVAS1 of zero as a definition of remission. (reviewed by Mukhtyar et al).19 Ignoring BVAS2 (persistent disease) underestimates true disease activity. This has therapeutic implications because persistent disease needs a different approach.16 Redundant and/or uncommon items have been identified in BVAS(v.2).20 Removal of those items would increase the feasibility of the modified clinical tool.

Aim

To validate a new version of BVAS for use in studies of systemic vasculitis.

Methods

The new clinical tool (see supplementary online appendix) was designed by consensus (face validity).20 The number of items was reduced from 66 in BVAS(v.2) to 56 in BVAS(v.3) either by omission or by merging with other items. The “persistent” boxes for each item were replaced by a single “persistent” box for the whole form. This box was marked, only if every disease manifestation was attributable to “persistent” disease. All items were treated as “new/worse” if any of them were “new/worse”. Disease manifestations were recorded if they had been active in the previous 4 weeks, and were directly attributable to vasculitis.

Weighting was largely unchanged, but new items were given weights by consensus. An extensive glossary and manual of operations was constructed. The BVAS(v.3) forms, training manual and glossary sheet are available online at the EUVAS website (http://www.vasculitis.org/disease.htm, accessed 13 July 2009).

Nineteen subjects completed 20 beginner-level training paper cases, and 14 of them, including all investigators recruiting patients for this study, completed 40 advanced-level paper cases. This paper-case exercise had two benefits; it trained each investigator in the use of BVAS(v.3) and provided information on interobserver reliability.

Inclusion criteria: Patients seen in outpatient clinics and hospital wards with a probable or definite diagnosis of systemic vasculitis, of any duration, at any stage of their disease, were recruited from nine UK centres: Birmingham (Birmingham City Hospital), Cambridge (Addenbrooke’s Hospital), Edinburgh (Western General Hospital), Norfolk (Norfolk and Norwich University Hospital), Nottingham (Nottingham University Hospital), Oxford (Oxford Radcliffe Hospitals and Nuffield Orthopaedic Centre), Reading (Royal Berkshire Hospital) and Westcliff-on-sea (Southend General Hospital). The investigators were rheumatologists (CM, DC, SD, PL, LY, RAL), nephrologists (OF, DJ, RJ), or allied health professionals with an interest in vasculitis (DB, CH, JH, AM).

Exclusion criteria: Patients with a probable or definite diagnosis of giant cell arteritis were excluded.

The study was approved by local ethics committees and all patients gave written informed consent. The data recorded on a standardised case record form included demographics, diagnosis, treatment in the previous 5 years, serum C-reactive protein (CRP), serum creatinine, antineutrophil cytoplasmic antibody (ANCA) status with titres and antibody levels, BVAS(v.3), treatment decision, physician’s global assessment (100 mm visual analogue scale), vasculitis activity index (five-point Likert scale, with 0 signifying remission and 4 suggesting maximum activity). Patients in Oxford and Edinburgh were also assessed using BVAS(v.2) at the same visit. Table 1 shows the treatment decisions classified into six ordered categories. When treatment decisions did not match the definitions, the category was jointly assigned by RAL and CM. The BVAS values for both versions were calculated on a Microsoft Excel based program developed by CM.

Table 1

Definitions of treatment decision categories

Convergent validity was assessed by examining the association of BVAS(v.3) with

  • BVAS1 subscore of BVAS(v.2) in patients who were assessed using both clinical instruments at the same hospital visit;

  • treatment decision (table 1);

  • nearest serum CRP level within 1 month of the consultation (when values were reported to be less than the lowest measurable value for that laboratory, the value was recorded as 0 mg/l for the purpose of statistical analysis);

  • physician’s global assessment;

  • vasculitis activity index.

The ability of the BVAS(v.3) to record persistent disease was assessed in patients with persistent disease on a follow-up assessment with BVAS(v.2). They were defined as having persistent disease if the BVAS2 subscore was higher than the BVAS 1 subscore.

Interobserver reliability (reproducibility) of BVAS(v.3) was examined in patients independently assessed by two observers on the same day. This was also assessed in the paper-case exercise.

Intraobserver reliability (repeatability) of BVAS(v.3) was examined in the subgroup of patients with a second assessment by the same observer within 9 days of the first assessment.

Sensitivity of BVAS to a change in disease state was assessed in a separate cohort of patients with a new diagnosis of ANCA-associated vasculitis. Thirty-nine patients were assessed at diagnosis and at 3 months using the BVAS(v.3), following treatment which was categorised as “major escalation” (table 1). It was expected that the patients would improve with the treatment and the change in BVAS was assessed as a marker of sensitivity to change in disease status.

Statistical analysis

The Spearman rank correlation coefficient was used to assess the association of BVAS(v.3) with subscores of BVAS(v.2), treatment decision, serum CRP levels, physician’s global assessment and vasculitis activity index. Reliability of the BVAS(v.3) was assessed using the intraclass correlation coefficient (ICC), and the linear weighted κ statistic for the organ-system subscores. Sensitivity to change was assessed by paired t test comparing BVAS(v.3) scores at 0 and 3 months. The statistical analysis was performed using SPSS 15.0 except for the linear weighted κ analysis which was performed using an online statistics package (http://faculty.vassar.edu/lowry/VassarStats.html, accessed 13 July 2009).

Results

Three hundred and thirteen patients with systemic vasculitis were recruited between 28 July 2004 and 11 December 2007. The mean (SD) age of the cohort was 55.0 (15.9); range 18–87 years. There were 162 women (52%), and 149 men (48%). Table 2 shows the demographics of the cohort. The median disease duration was 3.8 years (lower and upper quartiles 1.0, 8.7 years). The median BVAS(v.3) was 2 (for histogram see supplementary online figure 1).

Figure 1

Correlation between treatment decision and the Birmingham Vasculitis Activity Score (version 3) (BVAS(v.3)).

Table 2

Demographics of the cohort

Convergent validity

Of 313 patients, 302 had a recorded treatment decision. There was a correlation between the treatment decision and the BVAS(v.3) (Spearman’s correlation coefficient (rs) = 0.66, 95% CI 0.59 to 0.72) (fig 1). Subgroup analysis in 153 patients with Wegener’s granulomatosis revealed a similar correlation (rs = 0.72, 95% CI 0.64 to 0.79) (see supplementary online fig 2).

Figure 2

Correlation between the Birmingham Vasculitis Activity Score (version 3) (BVAS(v.3)) and the BVAS1 subscore.

One hundred and thirty-eight patients were assessed using the BVAS(v.2) and the BVAS(v.3). The correlation between the BVAS(v.3) and the BVAS1 subscore was strong (rs = 0.94, 95% CI 0.92 to 0.96) (fig 2). On follow-up, 19 patients were found to have persistent disease. Assessment with BVAS(v.2) and BVAS(v.3) at the same consultation demonstrated a strong correlation between BVAS(v.3) and the BVAS2 subscore of BVAS(v.2) (rs = 0.60, 95% CI 0.21 to 0.83).

One hundred and eighty-five patients had CRP levels available within 4 weeks of their BVAS assessment. There was a moderate correlation between BVAS(v.3) and CRP levels (rs = 0.43, 95% CI 0.31 to 0.54) (see supplementary online fig 3).

Figure 3

Birmingham Vasculitis Activity Score (version 3) (BVAS(v.3)) at diagnosis and at 3 months.

A strong correlation of BVAS(v.3) was demonstrated with a physician’s global assessment available for 307 patients (rs = 0.91, 95% CI 0.89 to 0.93) (see supplementary online figure 4), and a five-point Likert scale vasculitis activity index available for 304 patients (rs = 0.88, 95% CI 0.86 to 0.91) (see supplementary online fig 5).

Reliability

Reproducibility of BVAS(v.3) for the 20 basic-level paper cases completed by 19 observers was high with an ICC of 0.89 (95% CI 0.81 to 0.94). In the 40 advanced level paper cases completed by 14 observers the reproducibility was high with an ICC of 0.95 (95% CI 0.93 to 0.97).

In the 99 patients independently assessed by two observers on the same day the reproducibility of BVAS(v.3) was high with an ICC of 0.96 (95% CI 0.95 to 0.97). The reproducibility as measured by the linear weighted κ statistic was high for each of the nine organ-system subscores with the lowest weighted kappa being 0.78 (κ scores available as online supplementary table 1).

Repeatability of BVAS(v.3) in 39 patients assessed twice within 9 days, by the same investigator was high with an ICC of 0.96 (95% CI 0.92 to 0.97). A linear weighted κ statistic for the organ-system subscores showed good repeatability, minimum linear weighted κ was 0.75 (κ scores available as online supplementary table 1).

Sensitivity to change in disease status

Thirty-nine patients with a new diagnosis of ANCA-associated vasculitis (11 microscopic polyangiitis, 8 renal limited vasculitis, 20 Wegener’s granulomatosis) were assessed at diagnosis and 3 months using the BVAS(v.3). Each patient received treatment classified as “major escalation” (table 1). At diagnosis the mean (SD) BVAS(v.3) was 18.92 (6.06). Three months after treatment, the mean (SD) BVAS(v.3) fell to 2.03 (2.48) (fig 3). The mean fall in BVAS(v.3) of 16.9 (95% CI 14.8 to 18.9) was statistically significant (p<0.001 using paired t test). During this period the mean fall in the CRP levels was 40.0 (95% CI 15.4 to 64.7) mg/l.

Discussion

We have shown in this study that BVAS(v.3) has face validity, convergent validity with a number of parameters, and it is repeatable, reproducible and sensitive to changing disease states. In comparison with the previous validation studies of BVAS, this study is more robust, having larger patient numbers, wider breadth of vasculitis syndromes and a comparison with a higher number of parameters (table 3). Like previous validation exercises, we have not included assessment of giant cell arteritis. Giant cell arteritis is a distinct clinical syndrome with a limited rage of abnormalities as measured using BVAS items. This homogeneity of clinical manifestations would produce a limited range of scores and is not conducive to activity assessment using BVAS.

Table 3

Comparison of this study with other validation studies of different versions of the Birmingham Vasculitis Activity Score (BVAS)

The BVAS(v.3) was designed by consensus of a multispecialty group of vasculitis experts. It is a list of 56 items considered to be vasculitis manifestations with a numerical weight attached to each item, and each organ system has a ceiling score. These scores reflect the proportional importance of each manifestation and each organ system. This design of the BVAS(v.3) satisfies the conditions to meet “face validity”.

An ideal clinical tool should possess the ability to differentiate between diseases and/or disease states of interest. This ability of BVAS(v.3) can potentially be assessed in three different ways: first, the ability to differentiate between vasculitis and other conditions; second, the ability to differentiate active disease from damage; and lastly, the ability to differentiate between active disease, persistent disease and remission. The first two abilities are difficult to test owing to the rules governing the scoring of the BVAS(v.3). The marking of any manifestation on the BVAS sheet requires the doctor to be able to attribute its presence to active vasculitis. Strict application of this rule would automatically give all patients without vasculitis a score of zero, creating a spurious difference. In 39 patients we assessed the sensitivity of BVAS(v.3) to a change in disease status. In the absence of a valid external comparator, it is difficult to interpret a change in BVAS, but a fall of over 16 units is clinically meaningful. This improvement in clinical assessment was associated with a fall in CRP levels. There is no relationship between the magnitudes of change for the BVAS(v.3) and the CRP levels.

The BVAS(v.3) correlates only moderately with the CRP. This suggests that the inflammatory response is of value in making treatment decisions, but only in the correct clinical context. For example, the subject with the highest BVAS in our study (BVAS = 37) had active disease in five organ systems but had a CRP of <10 mg/l. Potential causes for this moderate correlation include—generation of CRP due to causes other than vasculitis, and treatment in the preceding 4 weeks producing a spuriously low CRP at the time of recruitment. A previous study did not find any correlation between CRP and disease activity.21 This study demonstrates an excellent correlation between the physician’s global assessment and the BVAS(v.3) score. This result is consistent with the results of Stone 2001,11 but not with those of Luqmani 1994 (table 3).9 This discrepancy can be explained by a fundamental difference in the way in which the exercise was carried out in the three studies. In Luqmani 1994, the physician’s global assessment and the BVAS assessment were independently done by two investigators. In this study, as in Stone 2001, the physician’s global assessment was performed by the same investigator after the BVAS assessment. It can be concluded that a physician’s global assessment in itself may not correlate significantly with disease activity, but a physician’s global assessment informed by the completion of a structured clinical interview and examination would have a good correlation with disease activity.

The BVAS(v.3) correlates with the treatment decision, but this correlation is not absolute. Of 67 patients who had a “major escalation” of their treatment, 12 had a BVAS(v.3) <5. This may be due to two reasons. First, initial treatment with prednisolone may have reduced disease activity. If the patient was recruited more than 4 weeks after the onset of symptoms, disease activity would be low but the start of an immunosuppressive maintenance regimen would be classified as “major escalation”, resulting in a discrepancy. Second, a rise of a few units may be a major flare in patients with certain vasculitis syndromes needing major escalation of treatment.

The BVAS(v.2) has been the “gold standard” for assessment of disease activity in clinical trials of systemic vasculitis. The excellent correlation between the two clinical tools makes the BVAS(v.3) a better tool owing to its ability to record persistent disease activity without the need for a separate subscore and its relative ease of use. It does not take any more time to complete than a structured clinical assessment. We have designed a Microsoft Excel based BVAS calculator which offers free online scoring of the BVAS at our website http://www.ndos.ox.ac.uk/research/rluqmani (accessed 13 July 2009). The calculator is a modification of the Microsoft excel programme developed for this study.

The BVAS(v.3), in common with all structured clinical assessments in vasculitis requires initial training, for which we have made the training manual freely available online at http://www.vasculitis.org/disease.htm. Training ensures that clinical observers, irrespective of their specialty, agree on what to score as directly attributable to active vasculitis.

The BVAS(v.3) is validated and ready for use in clinical trials. The focus of the BVAS has always been on clinical trials, but it may be of value in daily clinical practice. There is evidence that relapse of disease activity can occur in a previously unaffected organ system, so routine clinical examination and interview should be comprehensive and structured.22 The BVAS(v.3) serves as a checklist of items to screen for in daily practice. In the past, the complex scoring system has deterred routine clinical use of the BVAS, but the online BVAS calculator resolves that problem. With adequate prior training in the use of BVAS(v.3), the specialist and the general doctor can use the online calculator to convert disease activity into a tangible score giving them an accurate idea of the disease activity of their patient.

Disease-specific clinical assessment tools for vasculitis have been developed for Wegener’s granulomatosis (BVAS/WG) and Takayasu arteritis (Indian Takayasu Arteritis Score).11 23 These clinical tools may be more specific than BVAS when used for assessing disease activity in those conditions, although this has not been formally tested. However, early disease may not be accurately classifiable or the vasculitis syndrome may have overlap features.24 25 In these situations, the BVAS(v.3) may be used in preference to the disease-specific clinical assessment tools. The magnitude of activity as suggested by the BVAS(v.3) is not comparable across well-differentiated vasculitis syndromes, especially when they do not involve blood vessels of similar calibre, consistent with previously reported data.9 However, disease-specific modules could be developed to supplement BVAS, especially for diseases with a limited number of systems involved such as Takayasu arteritis. By retaining a generic set of items, it will be possible to compare core data between different vasculitis syndromes in order to provide more information on the long-term status of patients with systemic vasculitis, who now have a much improved life expectancy as a result of modern treatment. There may be a relationship between BVAS(v.3) and mortality but this could not be examined in our cross-sectional study.

Inevitably, with more extensive use, this version of BVAS may require editing and revision, as would be expected in any biological measurement.. However, BVAS (v3) currently represents a robust and useful tool for standard assessment of systemic vasculitis.

REFERENCES

View Abstract

Supplementary materials

Footnotes

  • ▸ Additional data are published online only at http://ard.bmj.com/content/vol68/issue12

  • Funding This study was funded by a grant from the Arthritis Research Campaign (project grant 16031). CM is funded by a project grant from the European League Against Rheumatism.

  • Competing interests None.

  • Ethics approval Approval from the multicentre research ethics committee.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.