The development of candidate composite disease activity and responder indices for psoriatic arthritis (GRACE project)
- Philip S Helliwell1,2,
- Oliver FitzGerald3,
- Jaap Fransen4,
- Dafna D Gladman5,
- Gerald G Kreuger6,
- Kristina Callis-Duffin6,
- Neil McHugh7,
- Philip J Mease8,
- Vibeke Strand9,
- Robin Waxman1,
- Valderilio Feijo Azevedo10,
- Adriana Beltran Ostos11,
- Sueli Carneiro12,
- Alberto Cauli13,
- Luis R Espinoza14,
- John A Flynn15,
- Nada Hassan16,
- Paul Healy17,
- Eduardo Mario Kerzberg18,
- Yun Jong Lee19,
- Ennio Lubrano20,
- Antonio Marchesoni21,
- Helena Marzo-Ortega1,
- Giovanni Porru22,
- Elvia G Moreta23,
- Peter Nash24,
- Helena Raffayova25,
- Roberto Ranza26,
- Siba P Raychaudhuri27,
- Euthalia Roussou28,
- Raphael Scarpa29,
- Yeong Wook Song30,
- Enrique R Soriano31,
- Paul P Tak32,
- Ilona Ujfalussy33,
- Kurt de Vlam34,
- Jessica A Walsh6
- 1Academic Unit of Musculoskeletal Medicine, University of Leeds, Leeds, UK
- 2Bradford Teaching Hospitals NHS Foundation Trust, UK
- 3St Vincent's University Hospital and University College Dublin, Dublin, Ireland
- 4Department of Rheumatology, The Radboud University Nijmegen Medical Centre, Nijmegen, The Netherlands
- 5Centre for Prognosis in the Rheumatic Diseases, University of Toronto, Toronto, Canada
- 6Department of Dermatology, University of Utah, Salt Lake City, Utah, USA
- 7Royal National Hospital for Rheumatic Diseases, Bath, UK
- 8Department of Rheumatology, University of Washington, Seattle, Washington, USA
- 9Division of Immunology and Rheumatology, Stanford University, Portola Valley, California, USA
- 10Department of Rheumatology, Federal University of Paraná, Rebouças – Paraná, Brasil
- 11Clinical Rheumatology, Clinical Rheumatology Hospital de la Policia, Bogota, Colombia
- 12University Hospital HUCFF of Federal University of Rio de Janeiro, Rio de Janeiro, Brasil
- 13Department of Medical Sciences, Policlinico of the University of Cagliari, Cagliari, Italy
- 14Section of Rheumatology, LSU Health Sciences Center, New Orleans, Louisiana, USA
- 15School of Medicine, Johns Hopkins University, Baltimore, Maryland, USA
- 16Rheumatology Department, Southend University Hospital, Westcliff-on-sea, UK
- 17Hutt Valley District Health Board, Lower Hutt, New Zealand
- 18School of Medicine, University of Buenos Aires, Buenos Aires, Argentina
- 19Department of Internal Medicine, Seoul National University Bundang Hospital, Seoul, Korea
- 20Department of Health Sciences, University of Molise, Campobasso, Italy
- 21Department of Rheumatology, Istituto Ortopedico G. Pini, Milan, Italy
- 22Department of Rheumatology, University of Cagliari, Cagliari, Italy
- 23St Paul Rheumatology, Eagan, Minnesota, USA
- 24Department of Medicine, University of Queensland, Maroochydore, Australia
- 25National Institute of Rheumatic Diseases, Piešt'any, Slovakia
- 26Rheumatology Unit, Universidade Federal de Uberlândia, Uberlândia, Brasil
- 27Division of Rheumatology, Allergy and Clinical Immunology, University of California Davis, Davis, California, USA
- 28Department of Rheumatology, Barking Havering and Redbridge University Hospitals NHS Trust, London, UK
- 29University Federico II, Naples, Italy
- 30Department of Internal Medicine, Seoul National University Hospital, Seoul, Korea
- 31Rheumatology Unit, Hospital Italiano de Buenos Aires, Buenos Aires, Argentina
- 32Clinical Immunology and Rheumatology, F4-105, Academic Medical Centre, University of Amsterdam, Amsterdam, The Netherlands
- 33National Health Center, Military Hospital, Budapest, Hungary
- 34Department of Rheumatology, University Hospitals Leuven, Leuven, Belgium
- Correspondence to Dr Philip S Helliwell, Leeds Institute of Molecular Medicine, Section of Musculoskeletal Disease, University of Leeds, 2nd Floor, Chapel Allerton Hospital, Harehills Lane, Leeds, LS7 4SA, UK;
- Received 13 January 2012
- Accepted 31 May 2012
- Published Online First 13 July 2012
Objective To develop new composite disease activity indices for psoriatic arthritis (PsA).
Methods Data from routine clinic visits at multiple centres were collected in a systematic manner. Data included all domains identified as important in randomised controlled trials in PsA. Decisions to change treatment were used as surrogates for high disease activity. New indices were developed by multiple linear regression (psoriatic arthritis disease activity score: PASDAS) and empirically, utilising physician-defined cut-offs for disease activity (arithmetic mean of desirability functions: AMDF). These were compared with existing composite measures: Composite Psoriatic arthritis Disease Activity Index (CPDAI), Disease Activity for PSoriatic Arthritis (DAPSA), and Disease Activity Score for rheumatoid arthritis (DAS28).
Results 161/503 (32%) subjects had treatment changes. Although all measures performed well, compared with existing indices, PASDAS was better able to discriminate between high and low disease activity (area under receiver operating curves (ROC)) curve with 95% CI: PASDAS 0.773 (0.723, 0.822); AMDF 0.730 (0.680, 0.780); CPDAI 0.719 (0.668, 0.770); DAPSA 0.710 (0.654, 0.766); DAS28 0.736 (0.680, 0.792). All measures were able to discriminate between disease activity states in patients with oligoarthritis, although area under the receiver operating curves (AUC) were generally smaller. In patients with severe skin disease (psoriasis area and severity index >10) both nonparametric and AUC curve statistics were nonsignificant for all measures.
Conclusions Two new composite measures to assess disease activity in PsA have been developed. Further testing in other datasets, including comparison with existing measures, is required to validate these instruments.
Psoriatic arthritis (PsA) manifests clinically in several ways, including arthritis, enthesitis, dactylitis, axial disease and skin/nail involvement. People with this condition may have one or all of these features. It follows that an assessment of disease activity in PsA should ideally record each feature that is present. To combine these assessments into a single composite index would further improve the efficiency of the measure.
Until recently, disease activity has been assessed in PsA randomised controlled trials (RCTs) by measures developed for rheumatoid arthritis (RA). The primary outcome measure adopted for all TNF-inhibitor trials has been the American College of Rheumatology 20% improvement (ACR20) criteria. An exception to this trend was the novel, albeit articular-based measure, which was developed for the Veterans Administration trial of sulfasalazine.1 These measures appear to function appropriately in the context of polyarticular PsA.2 ,3
In the last few years, composite measures of disease activity in PsA have been developed. The first was based on a treatment grid proposed by the Group for Research and Assessment of Psoriasis and Psoriatic Arthritis (GRAPPA). The Composite Psoriatic arthritis Disease Activity Index (CPDAI) assesses disease activity in five domains: skin, joint, enthesis, dactylitis and spine4 and, although comprehensive in coverage of domains, is subject to criticism for the empirical selection of cut-offs.5 Secondly, based on data derived from a large cohort, the Vienna group adopted the Disease Activity in REActive arthritis (DAREA)6 composite measure and reintroduced it as Disease Activity for PSoriatic Arthritis (DAPSA),7 which largely assesses the articular component of the disease. A performance comparison of CPDAI and DAPSA in the Psoriasis Randomised Etanercept Study in psoriatic Arthritis trial dataset confirmed the ability of the CPDAI to additionally measure changes in the skin and, therefore, to discriminate between two different doses of etanercept.8
Two types of composite indices may be envisioned. Responder indices, such as ACR20 in RA, measure changes in disease states with treatment interventions. A second type of index, such as the Disease Activity Score in RA9 ,10 measures both disease activity at a single time point and changes in disease activity after treatment interventions, thereby functioning both as a static measure of disease activity and a responder index. Ideally, a composite index should combine practicability and feasibility with validity and clinical relevance, and be easily applied in day-to-day treatment situations. Ideally, it would provide an absolute measure of disease activity, as well as response to therapy.
To develop such an instrument, GRAPPA designed a longitudinal study where data from routine clinic visits were collected in a systematic manner over 12 months. In this paper, the development of new measures from baseline data are reported.
All members of GRAPPA were invited, and 31 centres agreed to participate in this study. Centres were asked to provide data on consecutive routine clinic attendees to a minimum of 10 and a maximum of 40 patients. All patients granted informed consent, with ethical committee approval at each site. Data were collected at baseline (the first assessment), and 3, 6 and 12 months thereafter, recorded on case report forms (CRFs), and faxed or mailed to the coordinating centre in Leeds, UK. After review, any inconsistencies and missing data were referred to the originating centre for clarification.
Design and content of the CRF
Design and content of the CRF was by committee (see Acknowledgments), initiated at the 2006 Outcome Measures in Rheumatology (OMERACT 8) meeting.11 Consensus on the core domains to be assessed in RCTs in PsA was gained at the OMERACT 7 and 8 meetings, with >80% agreement.5 CRFs included existing instruments to assess each domain (table 1) as well as demographic and treatment data.
Assessing active disease by the ‘gold standard’
It was agreed that the ‘gold standard’ metric for active disease was a decision to change treatment at that clinic visit. The question was posed: ‘Are you changing this patient's medication today?' A change was equated to additions of medication, dose increases of current medications and/or changes to different medications. Reasons for medication changes and names of medications were further queried. If treatments were changed due to an adverse event, cases were excluded from the ‘changed medication’ group.
Comparator composite measures
Composite Psoriatic arthritis Disease Activity Index
This index measures disease activity in five domains: peripheral joints, skin, enthesitis, dactylitis and spine.4 A modification of the scoring system was used with the consent of the authors. This new scoring system graded severity in each category as 0 (none), 1 (mild), 3 (moderate) and 6 (severe). Cut-offs for each severity grade were not changed.
Disease Activity Index for Psoriatic Arthritis
This index measures disease activity in peripheral arthritis using: 68 tender and 66 swollen peripheral joint counts, patient global visual analogue scale (VAS), patient pain VAS, and C-reactive protein (CRP). The composite score is a simple sum of the scores.7
Disease activity score for RA (DAS28)
The DAS28 in RA includes a 28-joint tender and swollen counts, patient global VAS score, and either erythrocyte sedimentation rate or CRP.9 The score is calculated using weighting of the components, and ranges between 0 and 10.
In the development of the new measures, two approaches were used. The first simulated methods used in development of the Ankylosing Spondylitis Disease Activity Score.12 Initially, principal component analysis (PCA) was used to manage and reduce the variables into related components. Components with an eigenvalue of >1 were accepted. Factor loadings were then used as independent variables in a discriminant function analysis which used the decision to change treatment as the grouping variable. Finally, forward stepwise multiple linear regression analysis used the discriminant function previously obtained as the dependent variable, and original variables as independent variables.
From data collected, it was clear that a number of variables represented the same domain (table 1). For example, there were four enthesitis indices, four health-related quality-of-life measures, and six VAS scores. For enthesitis and health-related quality-of-life, a representative measure was selected for each of these domains based on univariate statistics comparing the metric in subjects with treatment changes and those without. Due to collinearity, some other variables were omitted—a correlation statistic (R) >0.85 determined the cut-off for this decision. Almost all variables were transformed to meet requirements of the analysis plan.
The second approach was that suggested by Fransen et al,13 where desirability functions were developed for variables deemed important in assessing disease activity based on core domains selected for PsA RCTs at OMERACT 8.11 Desirability functions for tender and swollen joint counts, health assessment questionnaire (HAQ) and patient global assessment of disease activity by VAS were derived using data gathered by an internet-based survey of GRAPPA members during development of the minimal disease activity score.14 Remaining functions (patient VAS for skin, patient VAS for joints, psoriasis area and severity index (PASI), and psoriatic arthritis quality-of-life index (PsAQoL) were developed with data obtained from 109 responses in a subsequent internet survey (85 rheumatologists and 24 dermatologists). Cut-offs were determined according to the median of responses (table 2), and used to transform each variable into linear functions ranging from 0 (totally unacceptable state) to 1 (normal). The eight transformed variables were then combined using the arithmetic mean (AMDF, arithmetic mean of desirability functions). The ability of new and existing measures to distinguish between active and inactive disease were compared at baseline with the Mann-Whitney test, and area under the receiver operating curves (ROC). ROC curves examine the ability of a measure to distinguish between two states, plotting sensitivity against (1—specificity). A straight line joining the bottom left (sensitivity=0, (1—specificity)=0) and top right corners would be obtained if the measure had no ability to discriminate between the two states, and would have an area of 0.5. A curve passing further away and to the left of this straight line approaches an area of 1.0 and better discriminates between groups.
Baseline characteristics are given in table 1. Patients numbering 503 were recruited at baseline. Participants were recruited from the following continents: Europe, 249; North America, 136; South America, 67; Australasia, 51. Only one centre, recruiting 17 patients, was primarily a dermatological centre, but many centres worked alongside dermatologists in combined clinics. At baseline, 178 subjects (35%) had a change in treatment, 17 (9.6%) due to adverse events or reductions in therapy; these latter subjects were therefore reclassified as ‘no treatment change’, resulting in 161 (32%) with treatment changes due to active disease.
Development of the psoriatic arthritis disease activity score (PASDAS)
PCA revealed seven components which approximated to the following domains: patient-reported measures (excluding mental component summary score (MCS) of the Medical Outcomes Survey Short form-36 (SF-36)), skin, peripheral joint counts, dactylitis, enthesitis, acute phase response and SF-36 (MCS). In the subsequent forward stepwise regression, two of the variables (patient and physician global VAS scores) accounted for approximately 90% of the total variance in scores. A hierarchical multiple regression analysis then considered these variables where both global VAS scores were entered in step 1, dactylitis, enthesitis, CRP, swollen joint count and SF-36 physical component scale in step 2, and finally tender joint count and SF-36 MCS (neither of which were significant in the forward stepwise regression) in step 3. Results of this regression analysis are presented in table 3. The variable coefficients determined the weighting used in the calculation of the PASDAS score, and a histogram of scores at baseline is shown in figure 1. They form a symmetrical distribution with a mean score of 4.3 (SD 1.7). As illustrated, MCS did not contribute to the model variance, and was therefore omitted from the final PASDAS score.
Development of the AMDF
Transformations were derived for the following variables: tender and swollen joint counts, HAQ, patient VAS for global assessment, patient VAS for skin, patient VAS for joints, PASI and PsAQoL, as indicated in the Statistics section. Individual scores were combined as the arithmetic mean. A histogram of the scores for this composite measure at baseline is presented in figure 2. Scores were positively skewed with a mean of 0.69 (SD 0.19). The distribution of scores toward the top end of the scale (1.0) reflected a generally good clinical state of this cohort at baseline.
Comparison of instruments at baseline
Instruments were examined for their ability to discriminate between subjects according to the decision to change treatment at baseline (table 4). In terms of z scores by Mann-Whitney testing and ROC curves, both PASDAS and AMDF performed better than other measures. Generally, measures that specifically included an assessment of the skin (AMDF and CPDAI) performed better than articular measures (DAPSA and DAS28), but not as well as the other composite index derived from baseline data in this study (PASDAS).
To examine the performance of all measures in different disease subgroups, data were analysed for subjects with oligoarthritis (<5 joints; N=266) and with severe skin involvement (PASI≥10; N=60), see supplementary online tables S1 and S2) As would be expected in subjects with oligoarthritis, scores for all instruments were lower and, generally, z and ROC scores were smaller, although remaining significant for the comparison between high and low disease activity. In the group with more severe skin disease, all measures reflected higher scores (indicating more severe skin and articular involvement for those changing treatment) but none of the measures could distinguish between treatment groups in this analysis.
Two novel composite disease activity measures for PsA have been developed in this study. The first, derived from baseline data using statistical techniques and modelling resulted in a weighted measure that included predominantly articular elements of disease (joint counts, enthesitis and dactylitis) as well as a generic quality-of-life measure, and both patient and physician global scores. If it can be assumed that the global scores encompass such elements as the skin and axial involvement, then this score covers the core domains identified for clinical trials in PsA. The second, derived empirically, and based on core domains chosen for assessment of PsA in RCTs, included assessments of both skin and joint involvement as well as a specific health-related quality-of-life measure. Both new instruments performed well, and overall better than existing measures in distinguishing ‘active’ from ‘inactive’ disease in the whole dataset, but were less able to do so in subgroups of oligoarthritis and patients with severe skin involvement.
One reason for this collaborative study was to develop a composite disease activity measure for PsA that could be represented by a single score. This approach has several advantages: comprehensive assessment of disease activity; as well as appropriately defined cut-offs for high and low disease activity, including remission and the ability to define change scores. These scores can, therefore, function as a measure of both disease activity and a responder index, as does the DAS28 in RA. In contrast with the DAS28 for RA, these measures cover a number of different manifestations of the disease and, thus, it may be argued that they are not unidimensional. However, all core domains identified for use in PsA clinical trials are included in these measures, and certain advantages may accrue from this. Such composite scores offer the advantage of ‘identifying’ a patient in need of further treatment when they may not qualify based on disease activity in a single component. As a composite measure, inclusive of all important manifestations of disease involvement, it can be easily applied in clinical practice as well as in regulatory RCTs. There are, however, potential disadvantages in this approach. First, a single score may underestimate improvements in some components, and deterioration in others. Second, some treatments may not work equally well for each of the disease manifestations, and a single composite score which does not demonstrate assessment of individual components will not detect a differential response. A possible solution to this is to report the individual components separately, as well as part of the composite score.
A limitation of this study is that most data were collected by rheumatologists, despite strenuous attempts to include dermatology centres. With participation of more dermatologists, it is likely that more decisions would have been made on the basis of severity of skin involvement and, quite possibly, a different outcome in terms of the proposed composite measure, with more emphasis on the skin. Although combining assessments of skin and joints represents an inclusive approach, there may be problems. For example, skin and joints do not always correspond in terms of disease activity and flares. A more practical issue is assessment of skin by rheumatologists, and joints by dermatologists, as often, expertise and confidence may be lacking. However, when specifically trained, dermatologists can reliably assess joints, and rheumatologists skin, as was demonstrated in the International Multi-centre Psoriasis And psoriatic arthritis Reliability Trial (IMPART) study.26 Perhaps the way forward should be closer working relationships between dermatologists and rheumatologists, with combined consultations for more complex cases.
In terms of the OMERACT filter, how do these measures perform, and what further studies will be required ?27 In terms of truth, it can be argued that an index which assesses all relevant domains of PsA will better reflect impact of the disease as a whole. AMDF and CPDAI certainly fulfil this criteria, and probably also the PASDAS, if it is accepted that the patient and, to a lesser extent the physician, global assessments will reflect involvement of the skin and spine. Discrimination will require further study utilising both existing and new interventional data. Although data exist on the reliability of individual measures within these composite indices,26 ,28 it will be important to generate further information on discrimination and responsiveness of the new indices. All measures require multiple assessments of articular and extra-articular features, and the two new measures require complex mathematical calculations to arrive at the single score. The latter problem is surmountable with web and calculator-based algorithms, but the former is time consuming. Experts in this field would argue that PsA is a complex multifaceted disease that requires more time for a complete clinical assessment. However, it is not clear how many rheumatologists and dermatologists without a special interest in PsA would routinely perform these assessments outside the clinical trial scenario.
The proposed and existing indices should now be further examined in databases from completed RCT treatment registries and applied in new interventional studies.
We thank Dr Robert Landewe and Dr Vern Farewell for advice with the statistical analysis. We would also like to acknowledge the following people who facilitated the study at the following centres; P Buiar, LF Hyurko, Federal University of Paraná, Brazil; R Valle, Hospital Militar, Bogota, Colombia; GG Krueger, University of Utah, USA; A Mathieu, Policlinico of the Univeristy of Cagliari, Monserrato-Cagliari, Italy; A Mumtaz, St. Vincent's University Hospital and University College Dublin, Ireland; D Christidis, Southend University Hospital, UK; G Ibrahim, J Curran, Bradford Teaching Hospitals NHS Foundation Trust, UK; A Castelli, N Fara, K Mendoza, S Medina, JM Ramos Mejía Hospital, Buenos Aires, Argentina; M Gingold, Royal National Hospital for Rheumatic Diseases, Bath, UK; L Coates, A Caperon, Leeds Teaching Hospitals NHS Trust, UK; J Rovensky, National Institute of Rheumatic Diseases, Piešt'any, Slovakia; L Ferreyra, Hospital Italiano de Buenos Aires, Argentina.
Contributors Study design and execution. The study was designed by Drs Helliwell, Gladman, FitzGerald, Cauli, McHugh, Soriano, and Strand after full discussion with the membership of GRAPPA. Study data and coordination were by Dr Helliwell and Ms Robin Waxman in Leeds, UK.
Funding a small amount of funding was provided by GRAPPA to cover some IRB permission fees in North America, and to provide some administrative support and database construction.
Ethics approval At each of the participating centres.
Provenance and peer review Not commissioned; externally peer reviewed.
Analysis and Writing The data was analysed and the paper was written by Dr Helliwell and modified by members of the Steering Committee (Helliwell, Kreuger, Kallis-Duffin, Gladman, Mease, FitzGerald, McHugh, Strand). All authors approved the final version of the paper.
Authors also involved in data collection PS Helliwell, O FitzGerald, DD Gladman, K Callis-Duffin, N McHugh, PJ Mease, VF Azevedo, A Beltran Ostos, S Carneiro, A Cauli, LR Espinoza, JA Flynn, N Hassan, P Healy, EM Kerzberg, YJ Lee, E Lubrano, A Marchesoni, H Marzo-Ortega, G Porru, EG Moreta, P Nash, H Raffayova, R Ranza, SP Raychaudhuri, E Roussou, R Scarpa, YW Song, ER Soriano, PP Tak, I Ujfalussy, K de Vlam, JA Walsh