Article Text

Download PDFPDF

  1. M. Schlereth1,
  2. A. Kleyer2,3,
  3. J. Utz1,
  4. L. Folle4,
  5. S. Bayat2,3,
  6. F. Fagni2,3,
  7. I. Minopoulou2,3,
  8. K. Tascilar2,3,
  9. J. Taubmann2,3,
  10. M. Uder5,
  11. T. Heimann6,
  12. J. Qiu1,
  13. G. Schett2,3,
  14. K. Breininger1,
  15. D. Simon2,3
  1. 1Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen, Germany
  2. 2Department of Internal Medicine 3 - Rheumatology and Immunology, Friedrich-Alexander Universität Erlangen-Nürnberg and University Hospital Erlangen, Erlangen, Germany
  3. 3Deutsches Zentrum Immuntherapie (DZI), Friedrich-Alexander Universität Erlangen-Nürnberg and University Hospital Erlangen, Erlangen, Germany
  4. 4Pattern Recognition Lab, Friedrich-Alexander Universität Erlangen-Nürnberg, Erlangen, Germany
  5. 5Institute of Radiology, Friedrich-Alexander Universität Erlangen-Nürnberg and University Hospital Erlangen, Erlangen, Germany
  6. 6Digital Technology and Innovation, Siemens Healthcare GmbH, Erlangen, Germany


Background Rheumatoid Arthritis (RA) Magnetic Resonance Imaging (MRI) scoring system (RAMRIS) [1] is used to manually assess severity of disease activity and monitor treatment response, but it is dependent on observer variability and is time-consuming. Deep learning techniques have the potential to improve the reproducibility and efficiency of RAMRIS scoring by automating the analysis of hand MRI scans, however, there are limited data on an automated assessment approach.

Objectives To investigate whether a deep neural network (DNN) can be trained to automatically detect erosion, synovitis, and oedema in RA patients using hand MRI scans and RAMRIS.

Methods We used 1,5 Tesla hand MRI (Siemens Magnetom Vida and Aera) from the BARE BONE trial, a prospective, single-arm, interventional, open-label, phase 4 trial (EUDRACT 2018-001164-32) in which RA patients were treated with baricitinib (4 mg/day) for 48 weeks. One of the objectives of BARE BONE was to assess the effect of baricitinib on joint damage and synovial inflammation. Participants of BARE BONE received hand MRI at week 0, 24, 48 following a standardized scanning protocol [2]. All images were scored according to RAMRIS. DNNs were applied on coronal T1 (pre/post contrast enhancement) and T2 MR images. 3-D landmarks for each location for RAMRIS scoring were identified and a region of interest (ROI) around each landmark was extracted to train a DNN. Three separate DNNs were trained, one for each of the RAMRIS subcomponents (erosion, synovitis, oedema). Each DNN is based on a ResNet-3D [3] architecture that was pretrained on a video classification task [4]. The networks were trained to predict the severity scores of each disease characteristic into three classes ranging from 0 (no pathological change) to 2 (high disease burden). The performance of the DNNs was evaluated using the area under the receiver operating characteristic curve (AUROC) and the area under the precision-recall curve (PR-AUC). Three-fold cross-validation were used and the network performance on a hold-out test set was evaluated.

Results In total, we obtained 212 coronal MR images with both T1 and T2 weighting from 30 RA patients (24 woman/6 men, age 53.5±12.6 years, disease duration 4.3±4.4 years). The overall RAMRIS score decreased from 20.6 (CI95% 14.4 to 27.8) to 18.3 (CI95% 11.5 to 26.5) at week 48. For the evaluation of erosions and oedema, 23 landmarks respectively were used per hand, and 7 landmarks for synovitis. In total 4608 landmarks for erosion and oedema were available, and for synovitis 1152 landmarks. The AUROC for predicting erosions was 86±2% with a PR-AUC of 83±4%. For the prediction of oedema the AUROC was 78±14% and PR-AUC was 83%±10%. Despite a low number of ROI for synovitis scoring, the respective AUROC was 60±4% and PR-AUC was of 69±3%.

Conclusion This proof-of-concept study demonstrated that fully automated extraction of synovitis, bone oedema, and erosion is feasible. In the future, our deep neural network approach may help to automatically assess MRI data of the hand in routine clinical practice and trials with high accuracy while keeping costs and human resources manageable.

References [1]Østergaard M et al. The Journal of Rheumatology 2017.

[2]Kemenes S et al. Annals of the Rheumatic Diseases 2022; 81:1320-1321.

[3]Hara K et al. arXiv:1711.09577, 2018.

[4]Kay W et al.

Figure 1.

Neural network pipeline. Region of interests required required for RAMRIS scoring are extracted from hand MR images. All region of interests required are fed into a ResNet 3D to detect erosion, synovitis and bone oedema, respectively.

Acknowledgements The study was supported by Lilly Deutschland GmbH and the DFG (FOR2886 PANDORA, CRC1181, CRC1483 EmpkinS). Additional funding was received by BMBF (project MASCARA), the ERC Synergy grant 4D Nanoscope, the IMI funded project RTCure, the Emerging Fields Initiative MIRACLE of the FAU Erlangen-Nürnberg. M.S., J.Q. and K.B. gratefully acknowledge support by d.hip campus - Bavarian aim. Infrastructural and hardware support was provided by the d.hip Digital Health Innovation Platform.

Disclosure of Interests Maja Schlereth: None declared, Arnd Kleyer Consultant of: Received consulting fees from Lilly Deutschland GmbH., Jonas Utz: None declared, Lukas Folle: None declared, Sara Bayat Consultant of: Received consulting fees from Lilly Deutschland GmbH., Filippo Fagni: None declared, Ioanna Minopoulou: None declared, Koray Tascilar Consultant of: Received consulting fees from Lilly Deutschland GmbH., Jule Taubmann: None declared, Michael Uder: None declared, Tobias Heimann: None declared, Jingna Qiu: None declared, Georg Schett: None declared, Katharina Breininger: None declared, David Simon Consultant of: Received consulting fees from Lilly Deutschland GmbH.

  • Artificial intelligence
  • Imaging
  • Rheumatoid arthritis

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.