Statistics from Altmetric.com
- CTD, connective tissue disease
- HCV, hepatitis C virus
- ROC, receiver operating characteristic
- SS, Sjögren's syndrome
Classification criteria often serve as diagnostic criteria. This is particularly true when the sensitivity and specificity of classification criteria are both close to 100%. In this case, classification criteria could be used as diagnostic criteria. This is rather unusual at the beginning of the disease, when the typical signs and symptoms are often lacking or are not entirely expressed. Classification criteria are therefore not perfect for use in diagnosis and a certain proportion of patients may be misclassified, particularly in the early stages of the disorder. Thus, classification cannot be considered the medical standard for a diagnosis and the expert doctor is the only person who can establish a definitive diagnosis for any individual patient. However, classification criteria for disease syndromes can be used to ensure the standardisation of the diagnosis in patients taking part in clinical studies, and to facilitate the analysis of results and the comparison of patients between institutions.1
Most of the rheumatic diseases lack a single distinguishing feature, however, and each disease is usually identified by the presence of a combination of clinical and laboratory manifestations. Therefore, the clinical observation of an expert clinician may be considered as the only available “gold standard” to define the diagnosis. This standard should consequently be used as the end point when calculating the sensitivity and specificity of the classification criteria for rheumatic diseases.
Classification criteria for most of the rheumatic disorders have been proposed and validated1–6 to establish the combination of disease features most useful for a definite diagnosis and to provide a uniform language for scientific communication.1 Sjögren's syndrome (SS) has been defined as an autoimmune epithelitis characterised by lymphocytic infiltration of exocrine glands and epithelia in multiple sites. The involvement of lachrymal and salivary glands results in the typical features of dry eye and salivary dysfunction (xerostomia). However, one third of the patients present with systemic extraglandular manifestations. Finally, SS can be seen alone (primary SS) or in association with other autoimmune rheumatic disease (secondary SS).7 Thus, the diagnostic approach to SS is rather complicated because it must include two different goals: firstly, assessment of the ocular and salivary components, and secondly, differentiation between the primary and secondary variants of the syndrome.
Different classification criteria sets have been suggested for SS, both before and during the First International Symposium on Sjögren's Syndrome held in Copenhagen in 1986,8–12 but none of these have been validated and universally accepted. Moreover, none of the proposed classification criteria sets include the same sequence of diagnostic tools for sicca components and for serological abnormalities.
THE CLASSIFICATION CRITERIA FOR SS ESTABLISHED BY THE EUROPEAN STUDY GROUP
In 1988 the European Study Group on Classification Criteria for SS began a multicentre study whose aims were (a) to validate a simple questionnaire for sicca symptoms; (b) to select the most sensitive and specific tests for the diagnosis of SS; (c) to define a set of classification criteria for this disorder; and, finally, (d) to validate this criteria set. Between 1988 and 1996 the goals of the study were reached, with different European centres joining the project during its various phases and providing a large number of patients and controls.
Twenty two centres from 11 countries which took part in the first part of the study provided 693 clinically defined cases, subdivided between patients with primary and secondary SS and controls. The preliminary diagnosis was based not on any single item or test, but on the clinical judgment of the observer, which was thus considered the “gold standard”. In this cohort of 693 patient and control cases, a protocol of selected diagnostic tests was then followed. A careful statistical analysis of the results allowed us to define a preliminary classification criteria set for SS.13
The second phase of the study was designed to validate and, if possible, improve the sensitivity and specificity of the preliminary classification criteria set. This goal was reached between 1993 and 1995 by testing the preliminary disease criteria set on a completely new population of clinically defined patients and controls. Twenty four centres from 13 countries contributed to this effort by enrolling 278 patient and control cases. The data from this second study were used to modify slightly the preliminary classification criteria set, and its sensitivity and specificity were marginally improved.14
The European classification criteria for SS have received broad acceptance by the scientific community; since their publication they have been used in a large number of clinical studies and have been cited in authoritative textbooks of internal medicine and rheumatology.
Some criticisms may be raised about this criteria set, however. The main criticism is that the combination of items I (ocular symptoms), II (oral symptoms), III (ocular signs), and V (salivary glands involvement), may also be met by patients with sicca symptoms, but without primary SS. In other words, the inclusion of patients who do not satisfy either criterion IV (focal sialoadenitis) or VI (anti-Ro/La antibodies), which are considered to be the most specific disease markers for SS, may introduce some misclassification bias. Another point that has been raised is that because two of the six criteria items are devoted to subjective complaints, and four of the six items must be met to classify a patient as having primary SS, patients with true primary SS, but without any subjective symptoms, might easily be misclassified.
THE AMERICAN-EUROPEAN STUDY GROUP ON CLASSIFICATION CRITERIA FOR SS
To overcome these objections and broaden the acceptance of the European classification criteria, the Sjögren's Syndrome Foundation proposed that a joint effort be undertaken by the European Study Group on Classification Criteria for SS and a group of American experts. The project obtained also a sponsorship by the European BIOMED Concerted Action BMH4-CT96–0595.
The Sjögren's Syndrome Foundation therefore organised and sponsored meetings between the two groups during the American College of Rheumatology's annual meetings in San Diego (1998), Boston (1999), and Philadelphia (2000), in Denver (September 1999), and during the VIIth International Symposium on Sjögren's Syndrome held in Venice (December 1999). During these meetings, the proposal that the European criteria could be accepted by the international rheumatology community as the most valid classification criteria set available for SS was thoroughly discussed.
To provide a solid basis for discussion, a more detailed analysis of the European database of the patients and controls collected during the third phase of the European Consensus Study14 was taken to the meetings. A receiver operating characteristic (ROC) curve15 of the revised criteria was constructed based on an analysis of 180 cases selected from a patient group provided by 16 centres from 10 European countries. The study group included 76 patients classified as having primary SS on the basis of the judgment of the clinician, 41 patients with different connective tissue diseases without clinical evidence of secondary SS, and 63 patients with sicca complaints but no SS (table 1).
The cases selected for the ROC curve analysis had all been subjected to the entire set of diagnostic procedures included in the European classification criteria. The curve was obtained by plotting the sensitivity and specificity values calculated for each different combination of positive tests (fig 1).
RESULTS OF THE ROC CURVE ANALYSIS AND CLASSIFICATION TREE PROCEDURE
The ROC curve analysis also allowed us to define the accuracy of different combinations of positive items in correctly classifying patients (true positive patient cases) plus controls (true negative control cases) with respect to the total number of cases included in the study group (fig 1). Based on this ROC curve analysis, the `C' point (positivity of any four out of the six items) and the `C*' point (positivity of four out of six items, with the exclusion of the cases that were negative for items IV and VI) were shown to have the same accuracy (92.7%), which was the highest among those obtained by different combinations of positive items. However, in comparison with the C point, the C* point had a lower sensitivity (89.5% v 97.4%) and a higher specificity (95.2% v 89.4%). Based on the concept that the main purpose of classification criteria is to standardise the selection criteria for patients with well defined disease who are to be included in study groups, in order to generate comparable results, the consensus group agreed that the item combination defined by the C* point, in view of its higher specificity, was more reliable for this purpose. Furthermore, the C* point introduces the concept of obligatory criteria, because every patient who was negative for both item IV (minor salivary gland biopsy) and item VI (anti-Ro/La autoantibodies) was defined as not having primary SS.
In comparison with the C* point, the `D' combination (positive results for three of the four objective criteria) showed a slightly lower accuracy (90.5%), with the same specificity but a slightly reduced sensitivity (84.2%). This was a consequence of the fact that four patients with clinically defined primary SS were not classified as having the disorder following the application of the D criteria combination. All four of these patients had responded in the affirmative to the questions about dry eye (item I) and dry mouth (item II). Two of the four patients had a positive lip biopsy (item IV) and positive results for the assessment of salivary glands (item V). The other two patients both had anti-Ro/La antibodies in their sera (item VI): and one of them tested positive for KCS while the other tested positive for salivary gland involvement. Nevertheless, the D combination can be considered a reliable set of criteria for the classification of patients with primary SS. It is worth noting that the D combination can also allow correct classification of patients with primary SS without subjective complaints.
The performance of the classification tree method (or the recursive partitioning procedure) was also tested on the same study group. The sequence in the classification tree procedure was created by examining every allowable split of each variable for each node. The most discriminant split was that which created two “daughter” nodes of progressively higher purity—that is, nodes which contained progressively larger proportions of either patients with primary SS or disease controls 16. This procedure proved quite valid and reliable in the classification of patients with primary SS, its sensitivity and specificity being 96.1% and 94.2%, respectively (fig 2).
In summary, the American-European Study Group agreed that the C* or D combination from the ROC curve should replace the previously proposed C combination14 in classifying patients with primary SS. The sequence of diagnostic tests suggested by the classification tree procedure may also be used to classify patients with primary SS, although it should be more properly used in a clinical-epidemiological survey.
OTHER MODIFICATIONS PROPOSED AND INCLUDED IN THE EUROPEAN CRITERIA SET
Definitions of the procedures to be used in performing the diagnostic tests included in the six item criteria set were initially established by the European Group which started the first multicentre study in 1989.17 The American-European Consensus Group subsequently decided that certain specifications must be added to the criteria sets in order to make the item definitions more precise and the tests more generally applicable. In particular, it was specified that Schirmer's I test should be performed without anaesthesia, and, because rose bengal is not available in many countries, other ocular dye scores (for instance, those performed using fluorescein stain for corneal surface and lissamine green for conjunctival surface) were suggested to replace it. Furthermore, the definition of item IV (histopathology) was slightly modified according to Daniels and Whitcher.18 Finally, it was more strictly indicated that the positivity of parotid sialography should be defined as the presence of diffuse sialectasis according to the scoring system of Rubin and Holt,19 and the positivity of salivary scintigraphy should be defined as delayed uptake, reduced concentration and/or delayed secretion of the tracer, according to the method proposed by Shall et al.20 (table 2).
The American-European Group also reached a consensus on a list of exclusion criteria (table 3). This list is quite similar to the one previously proposed by Fox et al in the so-called Californian criteria11 and then adopted by the European Study Group in the preliminary criteria set.13 Some modifications were made to the earlier list: (a) the category of “anticholinergic” drugs was used instead of “antidepressant, antihypertensive, parasympatholytic drugs and neuroleptic agents”; (b) “past head and neck radiation treatment” was added as an exclusion criterion; (c) sialoadenosis was deleted. Finally, it was decided to add hepatitis C virus (HCV) infection as an exclusion criterion, taking into account most of the data emerging from the current literature.
It is now well known that chronic HCV infection may mimic the clinical, histological, and immunological features of primary SS, although patients with HCV related SS are usually older, and have a lower prevalence of anti-Ro/La antibodies and parotid swelling, and a higher prevalence of hypocomplementaemia, cryoglobulins and liver disease.21 However, a significant number of patients with HCV related sicca syndrome actually meet the previously proposed European classification criteria.21 This convinced the American-European Consensus Group that HCV infection should be added to the exclusion criteria list.
CLASSIFICATION OF PATIENTS WITH SECONDARY SS
In previous studies it was suggested that patients with secondary SS could be correctly classified on the basis of the positivity of item I or item II, plus any two from among items III, IV, and V.13,14 The performance of this set of criteria was therefore compared by the study group with that of a set defined as the presence of at least two positive items from among items III, IV, and V (that is, excluding the subjective items).
The group used for this comparison comprised 72 patients clinically classified as having SS associated with another well defined disease (usually a connective tissue disease (CTD)), and 41 patients with CTDs but clinically classified as not having secondary SS. This study group was again derived from the European study population.14 The first set of criteria, which included subjective symptoms, showed a sensitivity of 97.2% (70/72 patients with secondary SS correctly classified) and a specificity of 90.2% (37/41 correctly classified controls—that is, patients with CTD without SS). The sensitivity of the second set of criteria was almost the same (98.6%—that is, 71/72 patients with secondary SS correctly classified), but its specificity was consistently decreased (80.5%—that is, 33/41 correctly classified patients with CTDs without SS). Therefore, the first procedure showed a better performance and it was decided that this was the most valid and reliable way of correctly classifying patients with secondary SS (table 3).
On the basis of our latest data, a modified classification criteria set (table 2), new rules for correctly classifying patients with primary and secondary SS, and a list of exclusion criteria were drafted (table 3) and approved by all the members of the American-European Consensus Group. The shared conclusion of the group is that the modified criteria set probably represents the best possible instrument presently available for the classification of patients with SS, as well as a useful starting point for future improvements. Obviously, this newly proposed classification criteria set for SS should be validated in further studies and in different groups of patients with SS and disease controls.
The following experts making up the European Community Study Group on Classification Criteria for Sjögren's Syndrome all participated in this multicentre study:
C Vitali, M Mosca, M Sciuto, S Bombardieri, Clinical Immunology and Rheumatology Units, University of Pisa, Pisa, Italy; HM Moutsopoulos, Department of Pathophysiology, Athens Medical School, Athens, Greece; J Coll, Department of Medicine, Hospital del Mar, Barcelona, Spain; R Gerli, Institute of Internal Medicine and Oncological Sciences, University of Perugia, Perugia, Italy; PY Hatron, Service de Médecine Interne A, Hôpital Huriez, Lille, France; L Kater, Section of Clinical Immunology, University Hospital Utrecht, The Netherlands; YT Konttinen, Institute of Biomedicine, Department of Anatomy, Helsinki, Finland; R Manthorpe, Sjögren's Syndrome Research Centre, University Hospital, Malmö, Sweden; O Meyer, Clinique Rhumatologique, Faculté Xavier Bichat, Paris, France; PA Ostuni, Division of Rheumatology, University of Padova, Padova, Italy; RA Pellerito, II Division of Medicine, Hospital Mauriziano, Torino, Italy; YL Pennec, Department of Internal Medicine, Brest University Medical School Hospital, Brest, France; SR Porter, Academic Department of Oral Medicine, Eastman Dental Hospital, London, UK; A Richards, Department of Oral Medicine, Surgery and Pathology, Bristol Dental Hospital and School, Bristol, UK; B Sauvezie, Clinical Immunology Unit, Hôpital Gabriel Mont Pied, Clermont Ferrand, France; M Schiødt, Hillerød Sjögren's Centre and Department of Oral Medicine and Oral Surgery, Hillerød, Denmark; Y Shoenfeld, Research Unit of Autoimmune Diseases, Sackler Faculty of Medicine, Tel Aviv, Israel; FN Skopouli, Department of Internal Medicine, Ioannina, Greece; J Smolen, 2nd Department of Medicine and Ludwig Boltzman Institute for Rheumatology, Vienna, Austria; F Soromenho, Serviço Medicina 2, Hospital Curry Cabral, Lisbon, Portugal; M Tishler, Rheumatology Unit, Ichilov Medical Centre, Tel Aviv, Israel; M Tomsic, University Medical Centre, Department of Rheumatology, Ljubljana, Slovenia; JP van de Merwe, Department of Immunology, Erasmus University and University Hospital, Rotterdam, The Netherlands; MJ Wattiaux, Hôpital Saint Antoine, Paris, France; CM Yeoman, Department of Oral and Maxillofacial Surgery, Royal Hallamshire Hospital, Sheffield, UK.
This study was partially supported by financial contributions from the European BIOMED Concerted Action BMH4-CT96–0595.
The American-European Consensus Group was convened and supported by the Sjögren's Syndrome Foundation, and funded in part by unrestricted educational grants from MGI PHARMA, Inc and the Allergan Foundation.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.