ReviewAutomated antinuclear immunofluorescence antibody screening: A comparative study of six computer-aided diagnostic systems
Introduction
The detection and measurement of autoantibodies against nuclear and cytoplasmic antigens (the so-called anti-nuclear antibodies — ANA) play a consolidate role for the diagnosis of systemic autoimmune rheumatic diseases (SARDs), such as systemic lupus erythematosus, rheumatoid arthritis, systemic sclerosis, Sjögren's syndrome, idiopathic inflammatory myopathies and systemic vasculitides. Indirect immunofluorescence (IIF) on human epidermoid laryngeal carcinoma cells (HEp-2 cells) is the most established method for ANA screening with the two-step diagnostic strategy for SARDs [1], [2], [3], [4]. The high sensitivity of ANA assessment by IIF, able to allow detection of more than 50 antibodies, makes this method an invaluable tool for the initial step of current diagnostic procedures for the detection of systemic autoantibodies [5], [6], [7].
However, the IIF method is burdened by some unfavorable features: the need for expert morphologists, the subjectivity of interpretation and a low degree of standardization and automation [4], [5], [8]. As a consequence, IIF is considered labor-intensive and prone to render bias.
During the last 15 years, the progressive increase of ANA test requests and volume of assays performed in clinical laboratories produced alternative solutions to the ANA-IIF test based on manual or automated monoplex and multiplex immunometric assays (enzymatic immunoassays — EIA; chemiluminescent immunoassays — CLIA; line-immunoassays — LIA), but literature reports demonstrated that these procedures do not provide the same analytical accuracy [5], [9].
The need for standardization of ANA testing continues to be a challenge, because its analytical variability continues to be high, without substantial improvement over time [10], [11], [12]. Recently, the biomedical industry has proposed technological solutions which might significantly improve the automation of the IIF procedure, not only in the preparation of substrates and slides, but also in microscope reading. This innovation is based on the principle of digitalization of fluorescent images, as an example of computer-assisted diagnosis, and on the classification of patterns using standardized approach (automated positive/negative screening and pattern interpretation) [13], [14], [15], [16], [17].
These systems are based on the use of automated microscopes, robotized slide trays, high sensitivity video cameras, and software dedicated to digital image acquisition and analysis. Currently, several commercial systems are available and have been evaluated in preliminary experimental studies on single devices [18], [19], [20], [21], [22], [23], [24], [25], [26], [27] with the purpose of assessing the reliability of automated IIF analysis as a standardized alternative for the conventional manual visual approach. Therefore, at present there are no studies comparing the different commercial technological platforms for automated ANA-IIF.
This study was undertaken to verify the level of accuracy of new automatic systems for the reading of ANA samples, specifically in discriminating between ANA-positive and ANA-negative samples. As a second objective, we analyzed the accuracy of these systems in pattern recognition, and checked whether there is correlation between levels of the analytic signal provided by the instruments and the titer obtained with manual IIF.
Section snippets
Patients and sera
We collected 104 ANA-positive sera and 40 ANA-negative sera. The preliminary selection of ANA-positive sera was made in eight laboratories of the Study Group on Autoimmune Diseases of the Italian Society of Laboratory Medicine (SIMeL) based on five main criteria: a) the source of sera (sera should be obtained from patients with a confirmed clinical diagnosis according to internationally accepted criteria); b) the type of pattern (in order to have a representative number of samples for each of
Positive/negative classification of ANA samples by automated interpretation
Sensitivity (measured on the 92 ANA-positive sera) and specificity (measured on the 34 ANA-negative sera) of the six systems at the cutoff adopted by the manufacturers are described in Table 3.
Of the 92 positive samples, there were a total of 18 false negatives resulting from the automatic readings: 2/92 for Helios, 3/92 for EUROPattern, 2/92 for Aklides, 6/92 for Nova View, 4/92 for Image Navigator, and 1/92 for G-Sight.
If we examine these 18 false negatives, 2 were those for which no definite
Discussion
This study is the first to compare the diagnostic accuracy of six systems for automated ANA-IIF reading on the same series of sera which included all the most clinically significant ANA-IIF patterns, with varying titer.
The scope of our study which, in total, produced and analyzed more than 5000 fluorescence tests (including manual and automated IIF testing) was: a) compare the results obtained through readings with the six automated instruments with the results obtained through manual ANA-IIF
Take-home messages
- •
Recently developed automated instruments for ANA reading and interpretation may significantly improve harmonization of the ANA immunofluorescence assay.
- •
We compared the diagnostic accuracy of six commercial systems for automated ANA-IIF reading on the same series of sera, and found that their efficiency in discriminating between positive and negative ANA samples is very high.
- •
The accuracy of pattern recognition, which is for now restricted to the most typical patterns, for the moment is limited.
- •
Acknowledgments
We gratefully acknowledge Euroimmun AG, Luebeck, Germany; Medipan, Dahlewitz, Germany; Inova Diagnostics, San Diego, CA; Aesku Diagnostics, Wendelsheim, Germany; A. Menarini Diagnostics, Florence, Italy; and Immuno Concepts, Sacramento, CA, for participating in this study.
References (29)
- et al.
Antinuclear antibody detection by automated multiplex immunoassay in untreated patients at the time of diagnosis
Autoimmun Rev
(2012) - et al.
Variability between methods to determine ANA, anti-dsDNA and anti-ENA autoantibodies: a collaborative study with biomedical industry
J Immunol Methods
(1998) - et al.
Human vs. machine: evaluation of fluorescence micrographs
Comput Biol Med
(2003) - et al.
Computer-assisted classification of HEp-2 immunofluorescence patterns in autoimmune diagnostics
Autoimmun Rev
(2003) - et al.
Automated subcellular location determination and high-throughput microscopy
Dev Cell
(2007) - et al.
Challenges of automated screening and differentiation of non-organ specific autoantibodies on HEp-2 cells
Autoimmun Rev
(2009) - et al.
Novel opportunities in automated classification of antinuclear antibodies on HEp-2 cells
Autoimmun Rev
(2011) - et al.
Detection of antinuclear antibodies by indirect immunofluorescence and by solid phase assay
Autoimmun Rev
(2011) - et al.
A comparative study on the reliability of an automated system for the evaluation of cell-based indirect immunofluorescence
Autoimmun Rev
(2012) - et al.
Antinuclear antibodies. A contemporary nomenclature using HEp-2 cells
J Autoimmun
(2010)