EXTENDED REPORT

Interobserver reliability of rheumatologists performing musculoskeletal ultrasonography: results from a EULAR "Train the trainers" course

A K Scheel¹, W A Schmidt², K-G A Hermann³, G A Bruyn⁴, M A D’Agostino⁵, W Grassi⁶, A Iagnocco⁷, J M Koski⁸, K P Machold⁹, E Naredo¹⁰, H Sattler¹¹, N Swen¹², M Szkudlarek¹³, R J Wakefield¹⁴, H R Ziswiler¹⁵, D Pasewaldt¹, C Werner¹⁶, M Backhaus¹⁷

¹ Department of Nephrology and Rheumatology, Georg-August-University Göttingen, Germany
² Medical Centre for Rheumatology Berlin-Buch, Berlin, Germany
³ Department of Radiology, Charité University Hospital, Berlin, Germany
⁴ Department of Rheumatology, Medisch Centrum Leeuwarden, Leeuwarden, The Netherlands
⁵ Department of Rheumatology, Ambroise Paré Hospital, UVSQ University, Boulogne Billancourt, France
⁶ Department of Rheumatology, Universita degli Studi di Ancona, Ospedale A. Murri, Jesi (Ancona), Italy
⁷ Department of Rheumatology, University La Sapienza, Rome, Italy
⁸ Department of Rheumatology, Mikkeli Central Hospital, Mikkeli, Finland
⁹ Department of Rheumatology, Vienna General Hospital, University of Vienna, Austria
¹⁰ Department of Rheumatology, the Research Unit, and the Epidemiology Unit, Severo Ochoa Hospital, Madrid, Spain
¹¹ Department of Rheumatology, Parkklinik, Bad Durkheim, Germany
¹² Department of Rheumatology, Medisch Centrum Alkmaar, Alkmaar, The Netherlands
¹³ Department of Rheumatology, University of Copenhagen, Hvidovre Hospital, Denmark
¹⁴ Academic Department of Musculoskeletal Medicine, Leeds General Infirmary, UK
¹⁵ Department of Rheumatology and Clinical Immunology, Inselspital, Bern, Switzerland
¹⁶ Department of Medical Statistics, Georg-August-University Göttingen, Göttingen, Germany
¹⁷ Department of Rheumatology and Clinical Immunology, Charité University Hospital, Berlin, Germany

Correspondence to:
Dr A K Scheel
Department of Medicine, Nephrology and Rheumatology, Robert-Koch-Strasse 40, D-37075 Göttingen, Germany; ascheel{at}gwdg.de

Objective: To evaluate the interobserver reliability among 14 experts in musculoskeletal ultrasonography (US) and to determine the overall agreement about the US results compared with magnetic resonance imaging (MRI), which served as the imaging "gold standard".

Methods: The clinically dominant joint regions (shoulder, knee, ankle/toe, wrist/finger) of four patients with inflammatory rheumatic diseases were ultrasonographically examined by 14 experts. US results were compared with MRI. Overall agreements, sensitivities, specificities, and interobserver reliabilities were assessed.

Results: Taking an agreement in US examination of 10 out of 14 experts into account, the overall for all examined joints was 0.76. Calculations for each joint region showed high values for the knee (1), moderate values for the shoulder (0.76) and hand/finger (0.59), and low agreement for ankle/toe joints (0.28). Values for bone lesions, bursitis, and tendon tears were high ( = 1). Relatively good agreement for most US findings, compared with MRI, was found for the shoulder (overall agreement 81%, sensitivity 76%, specificity 89%) and knee joint (overall agreement 88%, sensitivity 91%, specificity 88%). Sensitivities were lower for wrist/finger (overall agreement 73%, sensitivity 66%, specificity 88%) and ankle/toe joints (overall agreement 82%, sensitivity 61%, specificity 92%).

Conclusion: Interobserver reliabilities, sensitivities, and specificities in comparison with MRI were moderate to good. Further standardisation of US scanning techniques and definitions of different pathological US lesions are necessary to increase the interobserver agreement in musculoskeletal US.

Abbreviations: CRP, C reactive protein; ESR, erythrocyte sedimentation rate; FSE, fast spin echo; MCP, metacarpophalangeal; MRI, magnetic resonance imaging; RA, rheumatoid arthritis; SL, slice; STIR, short inversion recovery; US, ultrasonography

Keywords: ultrasonography; interobserver reliability; joints; magnetic resonance imaging

CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    What's this?

This article has been cited by other articles:

• Filippucci, E, Aydin, S Z., Karadag, O, Salaffi, F, Gutierrez, M, Direskeneli, H, Grassi, W (2009). Reliability of high-resolution ultrasonography in the assessment of Achilles tendon enthesopathy in seronegative spondyloarthropathies. Ann Rheum Dis 68: 1850-1855 [Abstract] [Full Text]  
• Funck-Brentano, T., Etchepare, F., Joulin, S. J., Gandjbakch, F., Pensec, V. D., Cyteval, C., Miquel, A., Benhamou, M., Banal, F., Le Loet, X., Cantagrel, A., Bourgeois, P., Fautrel, B. (2009). Benefits of ultrasonography in the management of early arthritis: a cross-sectional study of baseline data from the ESPOIR cohort. Rheumatology (Oxford) 48: 1515-1519 [Abstract] [Full Text]  
• Scire, C. A., Montecucco, C., Codullo, V., Epis, O., Todoerti, M., Caporali, R. (2009). Ultrasonographic evaluation of joint involvement in early rheumatoid arthritis in clinical remission: power Doppler signal predicts short-term relapse. Rheumatology (Oxford) 48: 1092-1097 [Abstract] [Full Text]  
• D'AGOSTINO, M.-A., CONAGHAN, P. G., NAREDO, E., AEGERTER, P., IAGNOCCO, A., FREESTON, J. E., FILIPPUCCI, E., MOLLER, I., PINEDA, C., JOSHUA, F., BACKHAUS, M., KEEN, H. I., KAELEY, G., ZISWEILER, H.-R., SCHMIDT, W. A., BALINT, P. V., BRUYN, G. A.W., JOUSSE-JOULIN, S., KANE, D., MOLLER, I., SZKUDLAREK, M., TERSLEV, L., WAKEFIELD, R. J. (2009). The OMERACT Ultrasound Task Force -- Advances and Priorities. The Journal of Rheumatology 36: 1829-1832 [Abstract] [Full Text]  
• Bruyn, G A W, Naredo, E, Moller, I, Moragues, C, Garrido, J, de Bock, G H, d'Agostino, M-A, Filippucci, E, Iagnocco, A, Backhaus, M, Swen, W A A, Balint, P, Pineda, C, Milutinovic, S, Kane, D, Kaeley, G, Narvaez, F J, Wakefield, R J, Narvaez, J A, de Augustin, J, Schmidt, W A (2009). Reliability of ultrasonography in detecting shoulder disease in patients with rheumatoid arthritis. Ann Rheum Dis 68: 357-361 [Abstract] [Full Text]  
• Wakefield, R J, Freeston, J E, O'Connor, P, Reay, N, Budgen, A, Hensor, E M A, Helliwell, P S, Emery, P, Woodburn, J (2008). The optimal assessment of the rheumatoid arthritis hindfoot: a comparative study of clinical examination, ultrasound and high field MRI. Ann Rheum Dis 67: 1678-1682 [Abstract] [Full Text]  
• Albrecht, K., Grob, K., Lange, U., Muller-Ladner, U., Strunk, J. (2008). Reliability of different Doppler ultrasound quantification methods and devices in the assessment of therapeutic response in arthritis. Rheumatology (Oxford) 47: 1521-1526 [Abstract] [Full Text]  
• Naredo, E, Bijlsma, J W J, Conaghan, P G, Acebes, C, Balint, P, Berner-Hammer, H, Bruyn, G A W, Collado, P, D'Agostino, M A, de Agustin, J J, de Miguel, E, Filippucci, E, Grassi, W, Iagnocco, A, Kane, D, Koski, J M, Manger, B, Mayordomo, L, Moller, I, Moragues, C, Rejon, E, Szkudlarek, M, Terslev, L, Uson, J, Wakefield, R J, Schmidt, W A (2008). Recommendations for the content and conduct of European League Against Rheumatism (EULAR) musculoskeletal ultrasound courses. Ann Rheum Dis 67: 1017-1022 [Abstract] [Full Text]  
• Violante, F S, Bonfiglioli, R, Graziosi, F, Caso, A, Isolani, L, Fiorentini, C, Mattioli, S (2007). Potential of ultrasonography for epidemiological study of work-related wrist tenosynovitis. Occup. Environ. Med. 64: 82-86 [Abstract] [Full Text]  
• Meenagh, G., Filippucci, E., Kane, D., Taggart, A., Grassi, W. (2007). Ultrasonography in rheumatology: developing its potential in clinical practice and research. Rheumatology (Oxford) 46: 3-5 [Full Text]  
• Koski, J M, Saarakkala, S, Helle, M, Hakulinen, U, Heikkinen, J O, Hermunen, H, Balint, P, Bruyn, G A, Filippucci, E, Grassi, W, Iagnocco, A, Luosujarvi, R, Manger, B, De Miguel, E, Naredo, E, Scheel, A K, Schmidt, W A, Soini, I, Szkudlarek, M, Terslev, L, Uson, J, Vuoristo, S, Ziswiler, H R (2006). Assessing the intra- and inter-reader reliability of dynamic ultrasound images in power Doppler ultrasonography. Ann Rheum Dis 65: 1658-1660 [Abstract] [Full Text]  
• Scheel, A K, Hermann, K-G A, Ohrndorf, S, Werner, C, Schirmer, C, Detert, J, Bollow, M, Hamm, B, Muller, G A, Burmester, G R, Backhaus, M (2006). Prospective 7 year follow up imaging study comparing radiography, ultrasonography, and magnetic resonance imaging in rheumatoid arthritis finger joints. Ann Rheum Dis 65: 595-600 [Abstract] [Full Text]  
• Naredo, E, Moller, I, Moragues, C, de Agustin, J J, Scheel, A K, Grassi, W, de Miguel, E, Backhaus, M, Balint, P, Bruyn, G A W, D'Agostino, M A, Filippucci, E, Iagnocco, A, Kane, D, Koski, J M, Mayordomo, L, Schmidt, W A, Swen, W A A, Szkudlarek, M, Terslev, L, Torp-Pedersen, S, Uson, J, Wakefield, R J, Werner, C, EULAR Working Group for Musculoskeletal Ultrasound, (2006). Interobserver reliability in musculoskeletal ultrasonography: results from a "Teach the Teachers" rheumatologist course. Ann Rheum Dis 65: 14-19 [Abstract] [Full Text]