Article Text
PDF
Abstract
Objectives To assess the reliability of the OMERACT ultrasound (US) definitions for the identification of calcium pyrophosphate deposition disease (CPPD) at the metacarpal-phalangeal, triangular fibrocartilage of the wrist (TFC), acromioclavicular (AC) and hip joints.
Methods A web-based exercise and subsequent patient-based exercise were carried out. A panel of 30 OMERACT members, participated at the web-based exercise by evaluating twice a set of US images for the presence/absence of CPPD. Afterwards, 19 members of the panel met in Siena, Italy, for the patient-based exercise. During the exercise, all sonographers examined twice eight patients for the presence/absence of CPPD at the same joints. Intraoberserver and interobserver kappa values were calculated for both exercises.
Results The web-based exercise yielded high kappa values both in intraobserver and interobserver evaluation for all sites, while in the patient-based exercise, inter-reader agreement was acceptable for the TFC and the AC. TFC reached high interobserver and intraobserver k values in both exercises, ranging from 0.75 to 0.87 (good to excellent agreement). AC reached moderate kappa values, from 0.51 to 0.85 (moderate to excellent agreement) and can readily be used for US CPPD identification.
Conclusions Based on the results of our exercise, the OMERACT US definitions for the identification of CPPD demonstrated to be reliable when applied to the TFC and AC. Other sites reached good kappa values in the web-based exercise but failed to achieve good reproducibility at the patient-based exercise, meaning the scanning method must be further refined.
- chondrocalcinosis
- ultrasonography
- osteoarthritis
Statistics from Altmetric.com
Introduction
Calcium pyrophosphate deposition disease (CPPD) is one of the most common arthropathies in the elderly, yet there is no specific treatment for this disease.1 Prevalence rates range from 4% to over 50%2–5 and increase with the age of the patient and the diagnostic method. The most recent EULAR recommendations on the diagnosis and terminology of CPPD endorsed ultrasound (US) as a promising diagnostic imaging modality for CPPD despite the use of synovial fluid analysis as the gold standard.6
Recently, an OMERACT special interest group for US in CPPD has been created with the aim to assess the potential utility of US in the diagnosis of CPPD and created for the first time definitions for US identification of calcium pyrophosphate (CPP) crystals in joints and soft tissues.7 Furthermore, the reliability of these definitions has been assessed in some sites (knee fibrocartilage and hyaline cartilage, patellar tendons, quadriceps tendons, Achilles tendons, triangular fibrocartilage of the wrist (TFC), synovial fluid of knee and wrist) demonstrating a good reliability in the menisci and hyaline cartilage of the knee and poor reliability at the other sites, specially at tendons and synovial fluid.7 However, previous imaging studies have demonstrated that CPP crystal deposition could also be found in other sites such as shoulders, hips or metacarpal-phalangeal joints (MCPs).8–11
The aim of this study was to assess the reliability of US by using the new OMERACT definitions for US identification of CPPD at the wrist (TFC), MCPs, acromioclavicular (AC) and hip joints and provide a comprehensive atlas of images of CPPD identified by US applying the new OMERACT US definitions for CPPD.
Patients and methods
Background and study design
The OMERACT US CPPD task force was created and held its first meeting during the American College of Rheumatology (ACR) congress in 2014. Since then, the group has produced a systematic literature review and meta-analysis12 on the use of US in the identification of CPPD and has created the US definitions for the identification of CPPD.7 First, reliability assessment of the definitions was then carried out in 2015 in Siena involving the fibrocartilage of the knee and wrist (TFC), the hyaline cartilage of the knee, the quadriceps/patellar/Achilles tendons and the synovial fluid of the knee and wrist. In the OMERACT US meeting group held during the EULAR congress in London (2016), the members of the group agreed that it would be very important for the validation process to extend the reliability tests in other joints, CPPD being a systemic disease, and to refine the scanning technique pertaining to difficult sites. A new workshop was organised in order to assess the reliability in the following sites: TFC, meniscus of the AC joint, hyaline cartilage of the MCP joints, labrum (HL) and hyaline cartilage (HCH) of the hip joints.
Following the OMERACT methodology,13 a web-based and a patient-based exercise was performed with the aim of testing the reliability of US in the detection of CPP deposits at difficult to evaluate joints.
Reporting of the results in this manuscript followed previously published guidelines.14 The study was notified and approved by the local ethics committee. All patients gave informed consent before participation in the workshop.
First step: the web-based exercise
Thirty rheumatologists from 14 different countries covering three continents (one from Colombia, one from Denmark, one from Egypt, three from France, one from Germany, nine from Italy, one from The Netherlands, two from Mexico, four from Romania, one from Serbia, two from Spain, one from Sweden, one from Switzerland and two from USA), experts in US and crystal-induced arthritis and members of the OMERACT US CPPD task force participated in the web-based exercise.
All participants were asked to send five images to the organisers of the workshop of the anatomical sites under examination (TFC, AC, MCP, HL and HCH) in order to prepare the web-based exercise. A set of 65 US images, equally distributed between the five different sites (13 images for each site), was then prepared based on the quality of the images (some of the scanned images were excluded), on the uniformity of the setting (trying to avoid excessive differences of the setting of the machines) and on the sites proposed (not all participants sent images of all sites). The sample was estimated to be the minimum size to accurately estimate kappa values significantly greater than 0.4, setting alpha at 0.05 and beta at 0.10.
Each participant rated the images according to a dichotomous score (presence/absence) by applying the definitions previously published.7 The US definition was available above every single image in order to avoid any misinterpretations.
Two weeks after the first assessment, all participants rated the same images again to assess the intraobserver reliability.
The web-based agreement exercise was carried out on a web-based platform (RedCap) that did not allow the submission of the survey in case of missing data. Only the facilitator and the epidemiologists of the study had access to the online data and were responsible for the upload and preparation of the Delphi rounds and the web-based exercise.
Second step: patient-based exercise
The patient-based exercise was held in Siena, Italy, in January 2017. Eight isolated stations were created with eight US machines (three Esaote, four GE and one Samsung) equipped with high frequency linear probes. The settings of the machines were created to better enhance calcific depositions and were tested and approved by the experts before the workshop. Each sonographer could modify only the basic functions (depth, gain, time gain control and frequency) in order to obtain the best possible image for CPP identification according to the patient’s physical characteristics.
Eight volunteer patients, five affected with CPPD (three men and two women; mean age 69.4±8.9 years) and three with osteoarthritis (two women and one man; mean age 58±8.2 years), as defined by synovial fluid analysis performed within 1 month before the study for routine clinical practice, participated in the patient-based study. Nineteen out of 30 ultrasonographers participated in this phase. Eight US stations were created for the workshop, four patients were sited and examined for the II and III MCP and the TFC bilaterally while in the remaining four stations the patients were examined for the hip and the AC joint in the supine position. Each sonographer examined all the patients and rated the presence/absence of CPP deposits. Four rheumatologists, experts in US and members of the local organising committee assisted the ultrasonographers during the whole procedure by collecting the datasheets, organising and timing the shifts at each station.
The US exam was performed according to a standardised sequence that was decided for each site, the day before the exercise during a briefing following the most recent guidelines of the EULAR.15 Briefly, the MCP joints’ hyaline cartilage (HC) was examined on the dorsal aspect of the hands with the fingers in maximum flexion to reveal a large portion of HC. Longitudinal and transverse scans were used. The AC joint was scanned on the longitudinal plane by placing the probe on the anterior aspect of the joint and sliding posteriorly making the most of the available acoustic window. Transverse scans were optional. The hip HC and HL were assessed only on the anterior aspect of the joint to reduce the discomfort of the patient as much as possible. Both structures were assessed mainly on the longitudinal axis, and transverse scans were optional. Also in this case, the sonographers were asked to use the entire available acoustic window and scan the largest possible portion of the structures under examination. Specifically, with regards to better identifying CPP deposits in the TFC, exam was done by sliding the probe over the structure, without lifting it, from the dorsal to the palmar aspect in longitudinal scanning and from proximal to distal for the transverse scanning. Dynamic scanning was also used by moving the wrist on the coronal plane and/or by pressing with the probe on the TFC when necessary to identify any pitfalls.
Each sonographer had 8 min to assess each joint of interest. After time expiration, the sonographer moved to the next station until every sonographer examined all patients. Power Doppler (PD) examination was not necessary for CPPD identification, but PD exam was allowed on sonographers’ judgement to better identify anatomical landmarks (vessels) or avoid pitfalls/artefacts (posterior enhancement of vessels that could mimic CPPD). Each sonographer rated the images according to a dichotomous score (presence/absence) by applying the previously published OMERACT definitions.7 The definitions were printed and provided to each sonographer before the exercise to avoid misinterpretations.
The procedure was repeated twice with the same patients the same day (morning and afternoon) to assess the intraobserver reliability.
Atlas of CPPD images
During the patient-based exercise (both the current one and the previously published7), sonographers were asked to save a representative image of each structure they examined. Three members of the panel reviewed all the stored images and choose four images that they considered to be the most representative images based on the OMERACT US definitions for CPPD at each site in order to create a US atlas.
Statistical analysis
Intraobserver and interobserver reliability were calculated using the kappa coefficient. Intraobserver reliability was assessed by Cohen’s kappa. Interobserver reliability was studied by calculating the mean kappa on all pairs (ie, Light’s kappa).16 Kappa coefficients were interpreted according to Landis and Koch.17 Kappa values of 0–0.20 were considered poor, 0.20–0.40 fair, 0.40–0.60 moderate, 0.60–0.80 good and 0.80–1 excellent. The percentage of observed agreement (ie, percentage of observations that obtained the same score), prevalence of the observed lesions and prevalence-adjusted bias-adjusted kappa were also calculated were also calculated.18 Analyses were performed using R Statistical Software (Foundation for Statistical Computing, Vienna, Austria).
Results
Web-based interobserver and intraobserver reliability exercise
All participants successfully completed both rounds of the web-based exercise. Inter-reader reliability, including both rounds, ranged from 0.56 (moderate agreement) reached for the HL to 0.75 (good agreement) achieved for the TFC and AC joint. Intraobserver reliability was higher in all sites varying from a minimum value of 0.73 (good agreement) for the HL to a maximum value of 0.85 (very good agreement) for the AC joint. Detailed results of the patient-based exercise are shown in table 1.
Web-based exercise results
Patient-based interobserver and intraobserver reliability
The patient-based exercise was successfully completed in two rounds of approximately 3 hours each: one in the morning and one in the afternoon of the same day. Interobserver reliability, including both rounds, ranged from 0.19 (poor agreement) for the HL to 0.82 (excellent agreement) for the TFC. Intraobserver reliability was higher in all sites and varied from 0.47 (moderate agreement) for the HL to 0.87 (excellent agreement) for the TFC. Detailed results of the patient-based exercise are presented in table 2.
Patient-based exercise results
US atlas of CPPD images
A US atlas of representative CPPD images of the various joints according to the new OMERACT definitions was created from selected images. More than 5500 files were reviewed, and the most representative images have been included in the atlas as shown in figure 1. The atlas also includes sites assessed in the previous exercise7 in order to provide a more comprehensive pictorial document that can be used as reference for the evaluation of CPPD by US.
Atlas of images of CPPD in the sites assessed by the OMERACT US CPPD group. The sites in white background achieved good reliability values during the exercises, while the sites in grey background did not. AC, acromioclavicular; CPPD, calcium pyrophosphate deposition disease; MCP, metacarpal-phalangeal joint; TFC, triangular fibrocartilage of the wrist; US, ultrasound.
Discussion
CPPD is considered to be one of the most frequent arthropathies of the elderly2 11; however, its true prevalence and incidence rates remain uncertain, and its impact on the health and disability of the persons affected is unknown. One of the major reasons for this discrepancy is that the only available methods for diagnosing CPPD until recently has been synovial fluid analysis and/or X-rays, which are invasive modalities and cannot be applied to large-scale epidemiological studies. Considering this aspect, one could also stipulate that the prevalence of CPPD is also underestimated because only patients with symptoms undergo these two diagnostic exams and a large proportion of our elderly population present the asymptomatic form and are never diagnosed. Furthermore, X-rays have been demonstrated to have a low sensitivity for the identification of CPP deposition,5 while synovial fluid analysis is not always performed, and CPP crystal are often missed contributing to the underdiagnosis.
Over the last decade, an increasing number of researchers have focused on the use of US in the identification of CPPD, highlighting the potential utility of this technique. The OMERACT US group acknowledged the growing evidence and interest of this application of US and founded the special interest group on CPPD with the objective to assess and standardise the potential use of the technique, following the OMERACT methodology. After creating for the first time the definitions for US identification of CPPD in various sites,7 the OMERACT CPPD subtask force gathered in Siena for the first reliability CPPD US exercise7 in 2015. In that occasion, due to time limitations for the relatively large number of sonographers and patients, it was chosen to assess the hyaline cartilage of the knee, the menisci, the tendons of the knee, the TFC and the synovial fluid. In that first exercise, the results showed good reliability for the menisci and the hyaline cartilage of the knee, but kappa values were low for the tendons, synovial fluid and for the wrist (TFC).
It was a common feeling at that time that high kappa values would be very difficult to reach for the tendons and the synovial fluid because of the big variety of conditions that could resemble CPP deposition according to the definition (ie, enthesophytes for the tendons, gout aggregates in tendons and synovial fluid and bubble airs in synovial fluid). However, there was enough evidence to convince the group that scanning the fibrocartilage and the cartilage should be sufficient to identify CPPD also in early stages and before the other diagnostic methods.12 19–21 Furthermore, the main sites of involvement in CPPD appear to be the joints, rich in fibrocartilage and hyaline cartilage such as knee, hip and wrist7 9 10 as suggested by the pathogenetic mechanisms and the role of chondrocytes in the formation of CPP deposition.22 23
Given this, the group agreed that it was important to assess the reliability and the potential of US in a larger number of joints and mainly to test again the wrist joint (TFC) as it seems to be one of the most frequently involved sites.8 During the second reliability exercise, a long briefing was held the day before the patient-based exercise in order to better define the scanning protocol and to test again the definitions on static images. The new EULAR standardised procedures for US15 were followed and considered sufficient for most of the joints plus the specific scanning of the triangular fibrocartilage (including dynamic scanning) as explained in the methods.
The results of the static exercise (both inter-reader and intra-reader) and the kappa values of the intra-reader agreement of the patient based exercise were generally good, but the inter-reader agreement of the patient-based exercise yielded good kappa values only for the TFC of the wrist (0.82: excellent) followed by the AC joint (0.52: moderate) and with other sites ranging from poor to fair, though not acceptable. That means that definitions were understood and applied, but the scanning technique probably needs to be further refined as differences in the kappa values can be only ascribed to different evaluation of the site under investigation by the sonographer at the time of examination.
However, as the most involved single sites in CPPD seem to be the wrist and the knee (TFC, menisci and HC of the knee), a most extensive evaluation and new exercises aimed to improve the kappa values in other sites could be superfluous if the diagnostic accuracy of the definitions is not good enough to allow their application in the clinical practice and/or for research purposes. For these reasons, the OMERACT US CPPD group agreed to move to the assessment of the diagnostic accuracy of the definitions at the knee before considering again the possibility to refine the scanning technique at sites that do not reach high reliability values.
This study was carried out by a group of rheumatology experts in musculoskeletal ultrasound (MSUS), but not all of them are experts in CPPD imaging. The members of the group adhere liberally to the initiative from the pool of sonographers of the OMERACT US group because of the interest on the disease even if they had not research experience previously on CPPD but only the skills that they developed with the daily clinical practice. The achievement of good kappa values between experts in CPPD and expert sonographers without particular interest before this study on CPPD suggests that the results could be reproducible also in the real world between expert sonographers. Furthermore, at the briefing session, the group decided to adopt the EULAR recommended scanning procedures15 that are freely available for every sonographer. Finally, in the US atlas of CPPD images included in this paper, the reader may find a comprehensive review of the US aspect of CPPD deposition in the different sites investigated during the two workshops even if they did not reach high reliability values. The images included in the CPPD US atlas portray some of the typical aspects of CPPD deposition according to the OMERACT definitions from a single CPP aggregate to diffuse deposition in order to provide a pictorial sample that can be used by both clinicians and researchers as a guide for diagnostic and research purposes.
In conclusion, US identification of CPPD is one of the most challenging applications of US in rheumatology. However, the lack of a harmless and non-invasive technique for the diagnosis of CPPD before the advent of US makes the research on US even more important. US could allow large epidemiological studies, follow-up of patients and, possibly, efficacy studies of drugs regarding either the inflammatory aspects of the disease or the joint damage/deposition extension due to CPPD.
Acknowledgments
The authors greatly acknowledge the contribution of General Electric Healthcare 27 Italy and Samsung Electronics Italia s.p.A for providing the US equipment for free for this study.
References
Footnotes
Handling editor Josef S Smolen
Contributors Planning of the work: GF, CAS, AA, GC and AI. Conducting of the work: all authors. Reporting of the work: GF, CAS, NSD, GAWB, MAD, EF, IMP, EN, CP, AMR, LT and AI.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient consent Detail has been removed from this case description/these case descriptions to ensure anonymity. The editors and reviewers have seen the detailed information available and are satisfied that the information backs up the case the authors are making.
Ethics approval Comitato Etico di Area Vasta Sud-Est (siena).
Provenance and peer review Not commissioned; externally peer reviewed.