Multicriteria decision analysis methods with 1000Minds for developing systemic sclerosis classification criteria

Sindhu R Johnson; Raymond P Naden; Jaap Fransen; Frank van den Hoogen; Janet E Pope; Murray Baron; Alan Tyndall; Marco Matucci-Cerinic; Christopher P Denton; Oliver Distler; Armando Gabrielli; Jacob M van Laar; Maureen Mayes; Virginia Steen; James R Seibold; Phillip Clements; Thomas A Medsger Jr; Patricia E Carreira; Gabriela Riemekasten; Lorinda Chung; Barri J Fessler; Peter A Merkel; Richard Silver; John Varga; Yannick Allanore; Ulf Mueller-Ladner; Madelon C Vonk; Ulrich A Walker; Susanna Cappelli; Dinesh Khanna

doi:10.1016/j.jclinepi.2013.12.009

Multicriteria decision analysis methods with 1000Minds for developing systemic sclerosis classification criteria

J Clin Epidemiol. 2014 Jun;67(6):706-14. doi: 10.1016/j.jclinepi.2013.12.009. Epub 2014 Apr 8.

Authors

Sindhu R Johnson¹, Raymond P Naden², Jaap Fransen³, Frank van den Hoogen⁴, Janet E Pope⁵, Murray Baron⁶, Alan Tyndall⁷, Marco Matucci-Cerinic⁸, Christopher P Denton⁹, Oliver Distler¹⁰, Armando Gabrielli¹¹, Jacob M van Laar¹², Maureen Mayes¹³, Virginia Steen¹⁴, James R Seibold¹⁵, Phillip Clements¹⁶, Thomas A Medsger Jr¹⁷, Patricia E Carreira¹⁸, Gabriela Riemekasten¹⁹, Lorinda Chung²⁰, Barri J Fessler²¹, Peter A Merkel²², Richard Silver²³, John Varga²⁴, Yannick Allanore²⁵, Ulf Mueller-Ladner²⁶, Madelon C Vonk³, Ulrich A Walker⁷, Susanna Cappelli⁸, Dinesh Khanna²⁷

Affiliations

¹ Division of Rheumatology, Department of Medicine, Toronto Western Hospital, University of Toronto, Ground Floor, East Wing, 399 Bathurst Street, Toronto, Ontario, Canada M5T 2S8; Division of Rheumatology, Department of Medicine, Mount Sinai Hospital, University of Toronto, Toronto, Ontario, Canada; Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario, Canada. Electronic address: Sindhu.Johnson@uhn.ca.
² Auckland City Hospital, Auckland, New Zealand.
³ Department of Rheumatic Diseases, Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands.
⁴ Rheumatology Centre, Sint Maartenskliniek, Nijmegen, The Netherlands.
⁵ Division of Rheumatology, Department of Medicine, St Joseph Health Care, University of Western Ontario, London, Ontario, Canada.
⁶ Division of Rheumatology, Department of Medicine, Jewish General Hospital, McGill University, Montreal, Quebec, Canada.
⁷ Rheumatology Department, University of Basel, Basel, Switzerland.
⁸ Department of Rheumatology AVC, University of Florence, Firenze, Italy; Department of BioMedicine, University of Florence, Firenze, Italy; Division of Rheumatology AOUC, Department of Medicine & Denothecentre, University of Florence, Firenze, Italy.
⁹ Centre for Rheumatology and Connective Tissue Diseases, Royal Free Hospital, London, UK.
¹⁰ Department of Rheumatology, University Hospital Zurich, Zurich, Switzerland.
¹¹ Dipartimento di Scienze Cliniche e Molecolari, Clinica Medica, Università Politecnica delle Marche, Ancona, Italy.
¹² Musculoskeletal Research Group, Institute of Cellular Medicine, Newcastle University, Newcastle upon Tyne, UK.
¹³ The University of Texas Health Science Center at Houston, Houston, TX, USA.
¹⁴ Division of Rheumatology, Clinical Immunology and Allergy, Department of Medicine, Georgetown University School of Medicine, USA.
¹⁵ Scleroderma Research Consultants, Avon, CT, USA.
¹⁶ Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, CA, USA.
¹⁷ Division of Rheumatology and Clinical Immunology, Department of Medicine, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA.
¹⁸ Servicio de Reumatología, Hospital Universitario 12 de Octubre, Madrid, Spain.
¹⁹ Department of Rheumatology, German Rheumatology Research Center, Leibniz Institute, Berlin, Germany.
²⁰ Division of Immunology and Rheumatology, Department of Medicine, Stanford University, Stanford, CA, USA; Department of Dermatology, Stanford University, Stanford, CA, USA.
²¹ Division of Clinical Immunology and Rheumatology, The University of Alabama at Birmingham, Birmingham, AL, USA.
²² Division of Rheumatology, The University of Pennsylvania, Philadelphia, PA, USA.
²³ Division of Rheumatology & Immunology, Department of Medicine, Medical University of South Carolina, SC, USA.
²⁴ Division of Rheumatology, Northwestern University Feinberg School of Medicine, Chicago, IL, USA.
²⁵ Rheumatology A Department, Paris Descartes University, Cochin Hospital, France.
²⁶ Department of Rheumatology and Clinical Immunology, Justus-Liebig University Giessen, Kerckhoff Clinic, Bad Nauheim, Germany.
²⁷ Scleroderma Program, Division of Rheumatology, University of Michigan, Ann Arbor, MI, USA.

Abstract

Objectives: Classification criteria for systemic sclerosis (SSc) are being developed. The objectives were to develop an instrument for collating case data and evaluate its sensibility; use forced-choice methods to reduce and weight criteria; and explore agreement among experts on the probability that cases were classified as SSc.

Study design and setting: A standardized instrument was tested for sensibility. The instrument was applied to 20 cases covering a range of probabilities that each had SSc. Experts rank ordered cases from highest to lowest probability; reduced and weighted the criteria using forced-choice methods; and reranked the cases. Consistency in rankings was evaluated using intraclass correlation coefficients (ICCs).

Results: Experts endorsed clarity (83%), comprehensibility (100%), face and content validity (100%). Criteria were weighted (points): finger skin thickening (14-22), fingertip lesions (9-21), friction rubs (21), finger flexion contractures (16), pulmonary fibrosis (14), SSc-related antibodies (15), Raynaud phenomenon (13), calcinosis (12), pulmonary hypertension (11), renal crisis (11), telangiectasia (10), abnormal nailfold capillaries (10), esophageal dilation (7), and puffy fingers (5). The ICC across experts was 0.73 [95% confidence interval (CI): 0.58, 0.86] and improved to 0.80 (95% CI: 0.68, 0.90).

Conclusions: Using a sensible instrument and forced-choice methods, the number of criteria were reduced by 39% (range, 23-14) and weighted. Our methods reflect the rigors of measurement science and serve as a template for developing classification criteria.

Keywords: Classification criteria; Conjoint analysis; Decision analysis; Forced-choice; Scleroderma; Sensibility; Systemic sclerosis.

Publication types

Research Support, N.I.H., Extramural
Research Support, Non-U.S. Gov't

MeSH terms

Biomedical Research / instrumentation*
Consensus
Decision Support Techniques*
Feasibility Studies
Humans
Reproducibility of Results
Scleroderma, Systemic / classification*
Scleroderma, Systemic / diagnosis

Abstract

Publication types

MeSH terms

Grants and funding