Objective: The present study was mainly aimed at investigating the interobserver and intraobserver reproducibility of ultrasound (US) results in the assessment of Achilles tendon enthesopathy in patients with seronegative spondyloarthropathies (SpA).
Methods: A total of 28 patients with a diagnosis of SpA according to the European Spondyloarthropathy Study Group criteria were included. The patient female/male ratio was 1.8 (18/10), mean age was 42 (range 25–75) years and mean disease duration was 9 (range 1–35) years. Mean (SD) Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) and Bath Ankylosing Spondylitis Functional Index (BASFI) scores were 32.4 (14.5) and 26.3 (9.2), respectively. Bilateral Achilles tendon US examinations were carried out independently by three investigators using a MyLab70 XVG (Esaote Biomedica, Genoa, Italy), equipped with a broadband 6–18 MHz linear probe. Each Achilles tendon was scanned for assessing the presence/absence of US findings indicative of enthesopathy according to the Outcome Measures in Rheumatoid Arthritis Clinical Trials (OMERACT) preliminary definition. The same findings were also scored on a 3-grade semiquantitative scoring system on which investigators reached a consensus prior to the study. Total additive scores per Achilles tendon were calculated.
Results: Moderate to excellent interobserver and intraobserver agreements were found for most of the US findings indicative of enthesopathy. Similar results were obtained using semiquantitative assessments, with weighted κ values estimating the interobserver and intraobserver agreements for soft tissue inflammation of 0.696 and 0.816, respectively and for tissue damage 0.711 and 0.901, respectively.
Conclusion: US assessment of Achilles tendon enthesopathy in patients with SpA, using the OMERACT preliminary definition, was found to be reliable. Bone irregularity and entheseal hypoechogenicity were the most difficult abnormalities to reach agreement on.
Statistics from Altmetric.com
Enthesitis is a characteristic feature of seronegative spondyloarthropathies (SpA)1 2 and some investigators believe it is the main pathology in the course of SpA.3 4 Entheseal involvement is often oligosymptomatic and its prevalence may be underestimated by clinical examination.5 6 7 8
Ultrasonography (US) is a non-invasive, inexpensive and sensitive imaging modality for assessing morphostructural abnormalities and blood flow changes at entheseal level. US evidence of enthesitis has been found in up to 98% of patients with SpA, the entheses of the lower limbs being the most frequently involved.6
Despite its potential to improve the clinical evaluation of entheseal involvement, relatively few studies have investigated US validity in patients with SpA, as most of the research being conducted is to assess synovitis in patients with rheumatoid arthritis.9 10 11 12 13
Since US is considered the most operator-dependent imaging modality, the present study was aimed at investigating the interobserver and intraobserver reproducibility of US results in the assessment of Achilles tendon enthesopathy in patients with SpA.
A total of 28 patients presenting at the Rheumatology Department of the Università Politecnica delle Marche, Ancona, Italy, with a diagnosis of SpA according to the European Spondyloarthropathy Study Group criteria were included.14 The patient female/male ratio was 1.8 (18/10), mean age was 42 (range 25–75) years and mean disease duration was 9 (range 1–35) years. All patients completed the Italian version of the Bath Ankylosing Spondylitis Disease Activity Index (BASDAI) for the activity disease assessment and the Bath Ankylosing Spondylitis Functional Index (BASFI) on the same day. Mean (SD) BASDAI and BASFI scores were 32.4 (14.5) and 26.3 (9.2), respectively.
The study was conducted according to local regulations and the Declaration of Helsinki. Informed consent was obtained from all patients. US examinations were performed independently by two investigators (SZA and OK), who attended the Rheumatology Department of the Università Politecnica delle Marche for a period of 1 month. Both the investigators were rheumatologists who attended a basic US course in January 2007 held in Ancona, Italy according to the standards of the European League Against Rheumatism (EULAR) sonography courses.15 They practised US in their Rheumatology Departments and interacted with the faculty of the course using a dedicated website (http://www.e-sonography.com) for a 6-month period before the start of the present study.16 Both investigators passed a basic competency examination carried out according to the test described in a recent report.17 One of them (SZA), investigator 1, had a special interest for SpA and spent a high percentage of her US practice scanning patients with SpA. A 4-h training session focused on US examination of the Achilles tendon was attended by the two investigators a few days prior to the study. This session was carried out under direct supervision of an experienced sonographer (EF), with the declared aim of reaching a consensus on US scanning technique and interpretation of US findings. The Achilles tendon insertion into the calcaneal bone was investigated because it can easily be explored by US and is the most frequently affected enthesis in patients with SpA.6 7 All patients included in the study were also independently scanned by an experienced sonographer (EF), and his US findings were used as standard reference. The intraobserver reliability of 1 sonographer (SZA) was calculated using representative images obtained in 14 consecutive patients with SpA (28 Achilles tendons). These images were rescored under blinded conditions at a minimum of 1 week after the initial recording.
The US examinations were performed using a MyLab 70 XVG device (Esaote Biomedica, Genoa, Italy), equipped with a broadband 6–18 MHz linear probe. Power Doppler settings were standardised with a pulse repetition frequency of 750 Hz, a colour-mode frequency of 9.1 MHz and low wall filters. The colour gain was increased to the highest value not generating power Doppler signals under the bony cortex.18 19 The patients were asked to take a prone position with the feet hanging out the examination table in neutral position for visualisation of the Achilles tendon.6 8 20 21 Bilateral Achilles tendon US examinations were carried out using a multiplanar scanning technique. Particular attention was paid to not apply transducer pressure to the anatomical structures under examination.
US images interpretation
The Outcome Measures in Rheumatology Clinical Trials (OMERACT) preliminary definition of enthesopathy was adopted for interpreting US images: “abnormally hypoechoic (loss of normal fibrillar architecture) and/or thickened tendon or ligament at its bony attachment (may occasionally contain hyperechoic foci consistent with calcification), seen in two perpendicular planes that may exhibit Doppler signal and/or bony changes including enthesophytes, erosions, or irregularity”.22 Table 1 describes in detail the US findings assessment. Figure 1 provides a representative pictorial example of the most relevant US findings indicative of enthesopathy. Achilles tendon thickness was measured by placing the callipers on the tendon margins at the insertion of the deeper tendon margin into the calcaneal bone. The threshold value for the identification of Achilles tendon thickening was 5.29 mm according to Balint et al.23 The presence of retrocalcaneal bursitis was defined by greyscale US using a multiplanar scanning technique aiming at detecting bursal enlargement. The maximal diameter obtained on longitudinal view was indicative of bursitis when higher than 2 mm.24 The power Doppler signal was scored on a 0–2 semiquantitative scale at its highest expression in three main anatomical structures: tendon, enthesis (preinsertional portion of the Achilles tendon; 1 cm long) and bursa.25 Power Doppler signals coming from within the bony cortex were not considered and interpreted as artefacts.26
Further analysis was conducted dividing the US findings into two main groups: those revealing soft tissue inflammation (susceptible of short-term changes) and those indicative of tissue damage (irreversible or requiring a longer time to change) (table 2).
Two total scores were calculated for each Achilles tendon according to the following rules: (1) a total score for soft tissue inflammation, which resulted from the sum of the scores assigned to the 7 US findings indicative of soft tissue inflammation, ranging from 0 to 7 with presence/absence data and from 0 to 14 with semiquantitative scores; (2) a total score for tissue damage, which resulted from the sum of the scores assigned to the 4 US findings indicative of tissue damage, ranging from 0 to 4 with presence/absence data and from 0 to 8 with semiquantitative scores.
Interobserver and intraobserver agreements were estimated by using κ statistics (unweighted κ for dichotomous assessment and linear weighted κ for semiquantitative scoring) and percentage of exact agreement. A κ value less than 0.20 was considered poor, between 0.21 and 0.40 fair, between 0.41 and 0.60 moderate, between 0.61 and 0.80 good, and between 0.81 and 1 excellent.27 The relationship between the total results of composite index obtained by the different sonographers was analysed using the method of Bland and Altman and calculated the 95% limits of agreement.28 Statistical analysis was performed with the MedCalc software package V.9.2 for Windows (MedCalc, Mariakerke, Belgium).
κ Values and percentages of exact agreement used to estimate the interobserver and intraobserver reliability for all US findings indicative of enthesopathy are reported in table 3 and table 4, respectively.
The levels of agreement on the single US findings of investigator 1 were generally higher than that obtained by investigator 2. The lowest κ values were found for both investigators in the detection and semiquantitative assessment of calcaneal bone irregularities. Thus, this finding was not included in the total additive score for tissue damage and was not assessed by investigator 1 during US examinations carried out for assessing intraobserver reliability.
Investigator 2 also found the detection of entheseal hypoechogenicity difficult to evaluate (unweighted κ value lower than 0.4). The same US finding was also difficult for investigator 1, being the category with the lowest κ value among those used to estimate the intraobserver reliability (table 4).
Weighted κ values estimating the interobserver agreement of total additive scores per Achilles tendon are reported in table 5. The weighted κ values of investigator 1 for intraobserver levels of agreement for total scores of soft tissue inflammation and tissue damage using semiquantitative scoring systems for each US findings were 0.816 and 0.901, respectively.
The Bland–Altman plots presented in fig 2 show a good agreement between the investigator 1 and the experienced sonographer. No systematic differences in their total additive scores for soft tissue inflammation and tissue damage were found.
Entheses of the lower limbs are frequently involved in patients with SpA. Among them, Achilles tendon insertion into the calcaneal bone was chosen as it was frequently affected and can be well explored using US.
This study demonstrates that two experienced rheumatologists with 6 months of ultrasound training were able to achieve high interobserver and intraobserver agreement rates for the detection of US findings indicative of enthesopathy according to the OMERACT preliminary definition. Moderate to excellent interobserver and intraobserver agreement (unweighted κ 0.497 to 0.913 and 0.462 to 1.000, respectively) was found, with the only exception for calcaneal bone irregularities (unweighted κ 0.207 and not calculated, respectively). Similar results were obtained for semiquantitative assessments of the same US findings. The levels of agreement were reported for each basic US finding, allowing the identification of the most difficult ones to be detected and scored. We believe this is important to facilitate learning and teaching, and to further develop the standardisation process.
The relatively low κ values obtained in the detection of entheseal hypoechogenicity appear to be affected by anisotropy. Bone profile assessment is also challenging because multiple enthesophytes may generate images that can easily be misinterpreted as bone erosions and vice versa. The identification of bone profile irregularities was poor or fair because even minimal changes can be detected by high frequency probe and it is not easy to reach a consensus on what is pathological and what is not. Further reasons for disagreement in the assessment of calcaneal bony cortex were related to the fact that some bone irregularities were variably misinterpreted as enthesophytes or bone erosions and US findings were not as distinct.
The results of the present study should be interpreted on the light of the fact that both investigators were trained with the same US equipment that was used for carrying out the study. This is an important factor highlighted in studies that assess reliability among experts.31 32 Of interest is that higher κ values were obtained by the investigator who had a special interest in SpA.
A limitation of the present study is related to the fact that the intraobserver reliability was assessed from 28 Achilles tendons in 14 patients. Since the results obtained from the same patient are generally correlated, a narrowing of the confidence intervals has to be considered.
Until now, only few studies aimed to determine the reproducibility of US in the assessment of entheseal involvement, and to the best of our knowledge no other investigations have been carried out to test interobserver and intraobserver reproducibility of US findings as described in the OMERACT preliminary definition of US enthesopathy using one of the latest generation US systems equipped with a very high frequency probe. Interobserver reliability was also determined by de Miguel et al in a recent study providing a comprehensive assessment of the validity issues on US evaluation of enthesitis. In that study, only the global interobserver agreement rate was reported, which is comparable to the result we obtained. Further investigations into this application are highly warranted.
Competing interests None declared.
Provenance and Peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.