Statistics from Altmetric.com
We read the comments by Pimentel-Quiroz et al with interest1 and agree it is important to test the performance characteristics of the 2022 ACR/EULAR Classification Criteria for the ANCA-associated vasculitides (AAV) in different populations around the world.2–4 These efforts are needed to understand whether regional differences in disease expression and clinical evaluation impact the utility of the new criteria in specific populations. The new classification criteria for AAV were developed within the largest observational cohort ever assembled in vasculitis,5 comprised of 4994 patients with different forms of vasculitis and 1997 patients with comparator diseases, recruited from 136 study sites in 32 countries. Although we assembled a large international cohort of patients with vasculitis, some regions of the world were under-represented, including Central and South America and Africa. With that in mind, Pimentel-Quiroz et al sought to validate the performance characteristics of the new classification criteria for granulomatosis with polyangiitis (GPA) and microscopic polyangiitis (MPA) in a cohort of patients from two centres in Mexico.
The purpose of classification criteria is to ensure that homogenous populations of patients with a particular disease are recruited into research studies.6 Excellent specificity is fundamentally important to minimise ‘false positives’ (ie, to exclude patients without the disease of interest from research studies in a particular disease). When the new criteria were developed, a threshold score for classification was chosen to maximise specificity. Importantly, Pimentel-Quiroz et al report excellent specificity of the new criteria in their cohort (GPA=96.2%; MPA=93.5%) that quite closely matches the specificity we reported (GPA=94.0%; MPA=94.0%). These findings confirm that the new criteria are fit for purpose to classify patients with GPA or MPA in their cohort.
Pimentel-Quiroz et al also report that the new criteria for GPA had a lower sensitivity (80.5%) when applied to their cohort compared with the sensitivity we reported (93.0%). Sensitivity is reduced when there are ‘false negatives’ (ie, patients who are clinically diagnosed with GPA but do not meet the threshold to be classified as GPA). While it is possible that regional variability in clinical features of GPA in patients from Mexico may have contributed to the lower sensitivity observed in this cohort, this finding may also be related to methodologic differences, including, importantly, the use of submitting physician’s diagnosis as the gold standard (Pimentel-Quiroz) versus expert panel review (2022 ACR/EULAR). Our restriction of the derivation cohort to cases rated by experts with a degree of certainty of the diagnosis of ‘moderately certain’ or ‘very certain’ by two independent reviewers allowed for exclusion of atypical cases of vasculitis. Cases with diagnostic uncertainty may not be appropriate to include in research studies, highlighting a key distinction between diagnostic and classification criteria.
Although diagnostic criteria intended for use in clinical practice should display near-perfect sensitivity and specificity to ensure that all patients with a particular disease are correctly diagnosed,7 classification criteria are designed to maximise specificity and specifically exclude atypical cases since homogeneity of study populations is preferred for research purposes. Therefore, lower sensitivity may be appropriate and expected in a heterogeneous cohort containing patients with clinical features that are not fully representative of a particular disease.
For the analyses of the 2022 ACR/EULAR criteria, we reported on a sensitivity analysis assessing the performance characteristics of the new criteria using, as the diagnostic gold standard, physician-submitted diagnosis rather than expert panel review. As expected, sensitivity was lower when the criteria were tested against physician-submitted diagnosis because this cohort contained more patients with a greater degree of diagnostic uncertainty. In the original report, the sensitivity of the new criteria in this secondary analysis was 83.8%, which is quite close to the 80.5% sensitivity reported by Pimentel-Quiroz et al in their cohort. Therefore, it would be important to further understand the reasons why some patients diagnosed with GPA in the Pimentel-Quiroz et al cohort did not meet the threshold for classification. It is possible this cohort included definable clinical differences related to regional variability of disease expression; however, it is also possible there was some degree of diagnostic uncertainty or atypical features of disease in the cases that were not correctly classified, making exclusion of such patients from a research study appropriate. Additionally, the sample sizes and confidence estimates were not reported by Pimentel-Quiroz et al, and small cohort size could substantively impact the precision of the results.
We are pleased that the specificity of the new criteria remained excellent in an independent population of patients with GPA and MPA from a population that was not well represented in our cohort. We encourage other investigators to continue to test the performance characteristics of the new 2022 ACR/EULAR Classification Criteria in additional populations of patients with vasculitis and anticipate that the criteria will function well and enable future research studies in AAV.
Patient consent for publication
Handling editor Josef S Smolen
Contributors All authors meet all of the following criteria for authorship: substantial contributions to the conception or design of the work; or the acquisition, analysis or interpretation of data for the work; drafting the work or revising it critically for important intellectual content; final approval of the version to be published. Agreement to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Funding JR has received honorarium from Roche and ChemoCentryx. RAW has received honoraria from Roche. RAL has received grants from Arthritis Research UK, GSK, MRC, University of Oxford Innovation Fund, Canadian Institutes of Health Research, The Vasculitis Foundation, Celgene and Vifor; consultancy fees and honoraria from Grunenthal, GSK, InflaRx, Medpace, MedImmune, Roche. PAM has received consulting fees from AbbVie, AstraZeneca, Biogen, Boehringer Ingelheim, Bristol-Myers Squibb, Celgene, ChemoCentryx, CSL Behring, Genzyme/Sanofi, GlaxoSmithKline, Genentech/Roche, InflaRx, Insmed, Janssen, Kiniksa, Sanofi and Sparrow, and research funds from Boehringer Ingelheim, Bristol-Myers Squibb, CaridianBCT, Celgene, ChemoCentryx, Genentech/Roche, GlaxoSmithKline, Kypha, American College of Rheumatology, European League Against Rheumatism, US National Institutes of Health, US Food and Drug Administration, The Patient-Centered Outcomes Research Institute and The Vasculitis Foundation, and royalties from UpToDate.
Provenance and peer review Commissioned; internally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.