Article Text
PDF
Abstract
Objectives Glucocorticoids used in the treatment of inflammatory rheumatic conditions can impact on health-related quality of life. An underpinning qualitative study developed a long-list of candidate items for a treatment-specific patient-reported outcome (PRO) measure. The objective of this paper is to determine scale structure and psychometric properties of the Steroid PRO.
Methods A cross-sectional survey of adults from the UK, USA, Australia and New Zealand, taking glucocorticoids for a rheumatic disease. Initial survey collected demographics, clinical information, 40 Steroid PRO candidate items and EuroQol-5 Dimensions- 5 levels (EQ-5D-5L). Follow-up, 3–5 days later, collected Steroid PRO candidate items and a condition-change (‘transition’) question. Analysis included Rasch measurement model, exploratory factor analysis (EFA), and hypothesis testing for discriminative validity, convergence validity and test–retest reliability.
Results Total responses 946: UK n=743 (79%); USA n=139 (15%); Australia/New Zealand n=64 (7%); mean age 57.6 (SD=13.6); 833 (88%) women. Participants with inflammatory arthritis n=197 (21%), connective tissue disease and/or vasculitis n=402 (42%), giant cell arteritis and/or polymyalgia rheumatica n=347 (37%). Twenty-five items were removed due to lack of fit to Rasch model. Of the remaining items, EFA suggested four subscales: Social impact (4 items); Impact on appearance (3 items); Psychological impact (5 items); Treatment concerns (3 items). Rasch modelling supported a four-subscale structure and total score, confirming construct validity and reliability. Hypothesis testing confirmed discriminant and convergence validity. Intraclass correlation coefficient (total score) was 0.809 demonstrating excellent (test–retest) reliability.
Conclusions The Steroid PRO is a 15-item, valid and reliable scale for measuring the impact of glucocorticoid therapy in people with rheumatic diseases.
- Glucocorticoids
- Patient Reported Outcome Measures
- Arthritis, Rheumatoid
- Systemic vasculitis
- Connective Tissue Diseases
Data availability statement
No data are available.
This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.
Statistics from Altmetric.com
- Glucocorticoids
- Patient Reported Outcome Measures
- Arthritis, Rheumatoid
- Systemic vasculitis
- Connective Tissue Diseases
WHAT IS ALREADY KNOWN ON THIS TOPIC
Glucocorticoids are widely used in treatment of the inflammatory rheumatic diseases but can impact on patients’ health-related quality of life (HRQoL).
WHAT THIS STUDY ADDS
This study determined the final scale structure and validates a treatment-specific patient-reported outcome measure for the impact of glucocorticoids from the patient perspective—the Steroid PRO.
The Steroid PRO provides consistent measures of HRQoL across patients with different demographics, rheumatic diseases and taking different doses of glucocorticoids.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
The Steroid PRO is a validated outcome measure for use in clinical trials to capture the impact of glucocorticoids from the patient perspective.
The Steroid PRO may also be used in clinical practice to aid understanding, communication and shared decision making between patients and their clinicians.
Introduction
Systemic glucocorticoids (GCs) are a class of steroid widely used in the autoimmune rheumatic diseases, to treat inflammatory arthritis, systemic vasculitis, connective tissue diseases and the crystal arthropathies. GCs are key in the management of life- and organ-threatening rheumatic diseases, but they have wide-ranging adverse effects which are of concern to patients and clinicians.1 2 Adverse effects include depression, anxiety, weight gain, skin thinning, insomnia, and risk of diabetes, osteoporosis, infection and cardiovascular disease.3
The international Outcome Measures in Rheumatology Glucocorticoid Working Group comprising patients, clinicians and methodologists in outcome measurement have identified the need for a patient-reported outcome (PRO) to measure glucocorticoid impact in clinical trials and practice.4 A 2010 European Alliance of Associations for Rheumatology working group has previously highlighted differences in patients’ and clinicians’ judgement of harm in relation to glucocorticoids, and recommended incorporation of the patients’ perspective.5 Underpinning semistructured qualitative interviews (n=60) have been performed with rheumatology patients from the UK, USA and Australia, receiving glucocorticoids (GCs) currently or within the last 2 years, for a range of rheumatic conditions. The interviews explored the impact of the medication on their symptoms and health-related quality of life (HRQoL).6 The themes identified were: physical symptoms, psychological symptoms, psychological impact of steroids, impact of steroids on participation, impact of steroids on relationships and benefits of steroids.6 These themes informed a long-form PRO measure (the draft Steroid PRO) with 40 candidate items, which was piloted with patient research partners (PRPs) and cognitive interviews with patients were conducted in the three countries.6 A linguistic translatability assessment was also performed (RWS Life Sciences). The overall study design employed a three-phase approach consistent with recommendations for best practice (Food and Drug Administration (FDA) guidance) for development of a PRO.7
We aimed to validate the Steroid PRO, in order to assess the impact of glucocorticoids on HRQoL in patients who are treated with glucocorticoids for rheumatic diseases. The objectives of this study were to determine final scale structure and measurement properties of the Steroid PRO.
Methods
Study design
This paper reports a cross-sectional validation study involving adult participants from the UK, USA, Australia and New Zealand. A steering committee comprising methodologists, clinicians and PRPs oversaw the study.
A web platform (Qualtrics XM) was used to design an anonymous online large-scale survey to collect patient responses at two time points. Timepoint 1 data were: (1) demographics—age, sex, country, ethnicity, educational level; (2) clinical information—diagnosis, glucocorticoid dose; (3) 40-candidate items for the Steroid PRO, developed in the previous qualitative study; (4) a generic measure of health state—EQ-5D-5L.8 Participants were given the option to receive a second (Timepoint 2) survey link 3–5 days after Timepoint 1, for test–retest reliability assessment. Timepoint 2 data were: (1) the Steroid PRO candidate items; (2) a condition change (‘transition’) question, “Overall, how are you NOW (in terms of the impact steroids are having on you) compared with 3–5 days ago (when you first answered the questionnaire)”.
Recruitment
Adults having current treatment with oral or intravenous glucocorticoids for a self-reported autoimmune rheumatic condition were eligible to participate. Participation in the study was voluntary and ethical approval was obtained from the study Sponsor, University of the West of England, Bristol (UWE REC Ref: HAS.21.09.011). Participants gave implied consent by completing the survey, and at the consent stage were given an option not to participate. Participants were recruited from the UK, USA, Australia and New Zealand. The survey was distributed via social media with support from patient groups, including a link to the study for further information and to access the survey. Study steering committee members were not eligible to participate.
Participants were assigned a random ID number by Qualtrics XM; this was used to link the Timepoint 1 and Timepoint 2 survey data. Participants had the right to withdraw from the study at any time during the survey, or up to 1 month following survey completion. To enable withdrawal requests, participants were given their ID number to quote if they wished to withdraw. Survey responses were excluded if the participant was aged <18 years; reported that they had not taken GCs in the last week; or if Timepoint 1 surveys were incomplete.
Data analysis
After descriptive analysis, iterative testing with Rasch measurement model and exploratory factor analysis (EFA) informed item reduction and established structural validity, reliability and unidimensionality of the final Steroid PRO.9–12 Item reduction decisions were based on clinical importance, lack of fit to the Rasch model (comparing the difference between observed responses and values expected by the model)10 and redundancy. Further evidence of validity of the Steroid PRO was established with hypothesis testing, by comparing Steroid PRO scores for participants receiving lower dose glucocorticoid (up to 10 mg) versus higher dose (>10 mg) (discriminative validity); and comparing scores of the Steroid Pro to EQ-5D-5L index (convergence validity).13 Evidence of reliability was established by (1) estimating the Steroid PRO internal consistency using the Person Separation Index (PSI, equivalent to Cronbach’s α) and (2) computing the intraclass correlation (ICC) between Timepoint 1 and Timepoint 2 Steroid PRO scores for patients who reported ‘no change’ in the impact of glucocorticoids on them, compared with 3–5 days previously (test–retest reliability). Furthermore, we estimated the minimum detectable change from the SE of measurement (SEm), obtained from the pooled SD (of the mean, Timepoint 1 and Timepoint 2) and ICC estimates (of average measures).13 14 All analyses were conducted using RUMM2030 (RUMM Laboratory, Perth, Australia) and IBM SPSS Statistics V.28.0.1.1 (IBM, Armonk, New York, USA) software.
Results
Recruitment
The large-scale online survey was distributed through patient groups via email, and social media (Twitter, Facebook and Instagram). The survey received 1748 initial page views, with 974 complete responses (all questionnaire items and demographics) at Timepoint 1. Twenty-five participants were excluded due to not having had glucocorticoids in the last week, and three removed due to spurious reporting of multiple rheumatic diseases. Complete responses analysed were therefore 946 at Timepoint 1 and 447 at Timepoint 2. No participants requested to withdraw from the study after survey completion.
Patient characteristics
Of the 946 participants with complete responses at Timepoint 1, 743 were from the UK, 139 USA and 64 Australia and New Zealand. Their mean age was 57.6 (SD=13.6), and 833 (88.1%) were women. In terms of occupation, 364 (38.4%) were employed, 347 (36.6%) retired, 154 (16.3%) disabled and 68 (8.2%) unemployed. The majority (616, 65%) had college/university degree. Participants had a diagnosis of inflammatory arthritis (n=197), connective tissue disease or vasculitis (n=402) and giant cell arteritis and/or polymyalgia rheumatica (n=347). Demographic characteristics are summarised in table 1. Full lists of diseases and participant ethnicities in online supplemental tables S1 and S2, respectively.
Supplemental material
Survey responses at Timepoint 1; demographic characteristics, disease groups
Validation
Of the 40 candidate questionnaire items, 14 were eliminated due to significant deviation from the Rasch model. A further 11 items were removed for floor effects and subsequent lack of fit to the Rasch model. The remaining 15 items had an adequate fit to the model.
Each of the 15 items had five response categories (never=0, rarely=1, sometimes=2, often=3 and always=4), which should reflect an ordered continuum from low to high impact on HRQoL (from 0 to 4). However, this structure displayed ‘disordered thresholds’ meaning that participants had difficulty to consistently discriminate between response categories. The two first categories (‘never’ and ‘rarely’) were amalgamated to constitute a four-category structure (never=0, sometimes=1, often=2 and always=3) which resulted in correctly ordered thresholds and a better fit to the Rasch model; χ2 (df) 155.93 (135); p=0.105 and adequate internal consistency, PSI=0.877. See Online supplemental table S3 for detailed Rasch analysis with individual items, comparing five category and four category response structure.
Principal component analysis suggested a structure with four subscales within the Steroid PRO: Social impact (4 items), Impact on appearance (3 items), Psychological impact (5 items) and Treatment concerns (3 items). PRPs were involved in naming the subscales (table 2).
Principal component analysis to determine the scale structure
Each of the four subscales were found to fit the Rasch model (table 3), and construct validity of the whole scale was confirmed; χ2=47.82 (df=36), p=0.899; reliability PSI=0.757 and only 2% of independent t-tests were significant in the Smith’s test of strict unidirectionality (p=0.022, 95% CI 0.008 to 0.036).
Rasch analysis with the four subscales
Targeting of the Steroid PRO
Figure 1 presents targeting of the items to persons. Figure 1A represents the pooled analysis of all patients, showing very good targeting of items to different impact (HRQoL) levels. Figure 1B represents the persons divided by the three condition groups: Inflammatory arthritis (n=194); connective tissue diseases (CTD) and vasculitis (n=398); and giant cell arteritis (GCA) and polymyalgia rheumatica (PMR) (n=341). This also demonstrated a good targeting of items across impact levels and the disease groups.
Person-item threshold distribution for the 15 items of the Steroid PRO (N=933). These graphs present the distribution of items: the x-axis is the logit score representing the interval scaling of the items according to the Rasch model, with −2 being no impact and 2 being high impact of corticosteroids. The lower part of each histogram is where individual items are located along the scale; the top part of histogram represents the number of people and their total Steroid PRO logit score. PRO, patient-reported outcome.
Test of item bias (invariance)
Following fit to the Rasch model, all subscales were confirmed to be invariant to age, gender, disease duration, glucocorticoid dose, disease state (active vs remission) and country (UK, Australia, New Zealand and USA). Based on disease groups, the Treatment concerns subscale displayed a uniform bias towards scoring higher impact in the GCA/PMR disease group compared with other disease groups (figure 1; online supplemental table S4 and figure S1). No differential item functioning (DIF) adjustment was necessary however, as this was shown to cancel in the top-down purification test.15
Validity with hypothesis testing
Discriminative validity
Most of the participants (n=696, 74.1%) reported to be taking a ‘low dose’ of glucocorticoid (up to 10 mg), while 243 (25.9%) reported to take a ‘high dose’ (over 10 mg). All Steroid PRO domain scores differed significantly between patients who self-reported taking a ‘low dose’ versus those taking a ‘high dose’, suggesting that the Steroid PRO had discriminative (known groups) validity (table 4).
Discriminative (known groups) validity for the four subscales of Steroid PRO
Convergence validity
The total Steroid PRO had moderate correlation with EQ-5D-5L index (rp=−0.550; 95% CI –0.593 to –0.504). Two subscales (Social and Psychological) had moderate correlation with EQ-5D-5L index while the other two subscales (Appearance and Treatment concerns) had weak correlation (table 5). Hypothesis of convergence validity was supported.
Convergence validity with EQ-5D-5L index
Test–retest reliability and minimum detectable changes
A total of 447 participants returned the Timepoint 2 (retest) Steroid PRO questionnaire. Compared with 3–5 days ago, 374 (78.4%) reported ‘no change’ in the impact of steroids; 6 (1.3%) ‘much better’; 53 (11.1%) ‘slightly better’; 42 (8.8%) ‘slightly worse’ and 2 (0.4%) ‘much worse’. All the 95% CI of the ICC estimates of the domain scores at Timepoint 1 and Timepoint 2 (3–5 days later), in those whose conditions had not changed, was between 0.751 and 0.911 indicating ‘good’ to ‘excellent’ reliability (table 6).
Test–retest reliability and minimum detectable changes
Discussion
This study has used modern psychometric methods to develop the Steroid PRO, the first generic measure of the impact of GC therapy on HRQoL for patients with a rheumatic disease. The Steroid PRO development has been underpinned by patient involvement at every stage, including in-depth qualitative and cognitive interviews with patients across the major rheumatic diseases, in the UK, Australia and the USA.6 The final Steroid PRO with its four subscales (Social impact, Impact on appearance, Psychological impact and Treatment concerns) has been designed to be highly relevant to patients. Data from the Steroid PRO were shown to have adequate fit to the Rasch measurement model, confirming its construct validity, and statistical sufficiency of the subscales and the total score,10 13 therefore providing accurate estimates of HRQoL due to GC therapy. The tool can thus be used in clinical trials, or practice, as a validated outcome measure.
The Steroid PRO items were shown to target well across different levels of HRQoL and provide consistent measures in patients with different personal characteristics (age, gender, education and occupation), cultures (UK, USA, Australia and New Zealand) and disease groups (inflammatory arthritis, connective tissue disease or vasculitis, and giant cell arteritis and/or polymyalgia rheumatica).
This is the first generic patient-reported outcome measure (PROM) for GC impact across the rheumatic diseases. The Glucocorticoid Toxicity Index is a measure developed for clinicians to use in the assessment of GC-specific outcomes.16 The Steroid PRO focuses on patients’ perceptions of steroid impact and salience which may differ from clinical outcomes or concerns.6 The Systemic Lupus Erythematosus (SLE) Steroid Questionnaire (SSQ)17 is a disease-specific GC impact PROM for SLE.10 The SSQ has also been designed using robust methodology, and covers similar domains, but it is tailored specifically for patients with SLE and has not been validated for use in patients with other inflammatory rheumatic diseases. The Steroid PRO thus addresses the need for an instrument with broader utility across diseases.
The strengths of this study include a large sample size with an excellent proportion of completed Timepoint 1 and Timepoint 2 responses to enable robust statistical analyses across the three countries and three broad disease groups. Also, as the draft items were developed from qualitative interviews with patients,6 our dataset for quantitative validation started with a set of good items which were further improved though item reduction, thus the Steroid PRO has high clinical relevance.
This study has three main limitations: First, recruitment through online patient groups may introduce self-selection and exclude patients who do not use online groups for support or those who do not have access to technology.18 This is an important limitation especially considering that patients with low health literacy are likely to experience a higher burden of disease19–21 and (by implication) steroid impact. Our sample indicates we had a broad range of ages, educational levels and employment status, although there was a high proportion of participants with degree-level education. There were more women who participated than men. This may reflect the underlying demographic distribution of the diseases themselves, which are usually more common in women,22 or it may be related to the recruitment method. We used several checks to protect data quality; for example, removing records where the survey was incomplete, or where the number or combination of diagnoses was judged to be clinically improbable. Second, we did not test responsiveness of the Steroid PRO as this will require a prospective study. However, given the satisfaction of the requirements of Rasch measurement model, confirming its construct validity, discriminative validity and reliability (also test–retest reliability), the Steroid PRO is likely to have responsiveness, but this need to be evaluated in future studies. Lastly, while the Steroid PRO worked well across four cultures (UK, USA, Australia and New Zealand), these are all English-speaking countries, therefore a cross-cultural validation will be required if the tool is to be used in other countries or for multinational comparisons. Linguistic translatability assessment has already been performed as part of the development of the initial candidate items, to ensure the wording and structure of items will be suitable for formal translation in the future.6
The Steroid PRO validation has established the SEm and the minimum detectable change, allowing an understanding of the difference in scores that will represent a real change (beyond measurement error). Discrimination between different HRQoL levels can be measured at either individual or group levels using the Steroid PRO. These parameters can be used in clinical trials to give an estimation of proposed study sample sizes. Therefore, the Steroid PRO will provide accurate assessment of GC impact in clinical trials, for example, when testing targeted steroid-sparing treatments and GC regimens.
In addition to its use as an outcome measure in clinical trials, the Steroid PRO could be used in clinical practice to facilitate communication between patients and clinicians about topics of high relevance to patients,23 both in remote and in-person consultations, and as an aid to support shared decision making when deciding on risks or benefits of different treatment regimens with individual patients.24
Qualitative studies on the impact of GCs in non-rheumatic conditions such as asthma and inflammatory bowel disease reported similar themes to those observed in our underpinning study (eg, impact on weight, appearance, sleep, mood and participation).1 6 25–28 Cross-condition validation could explore whether the Steroid PRO may also be acceptable and effective for use in non-rheumatic inflammatory conditions.
In conclusion, while GCs are widely used in the management of most autoimmune and rheumatic diseases, assessing their impact on HRQoL is important for both patients and clinicians. The final 15-item Steroid PRO has satisfied the strictest standards of measurement, thus providing an accurate measure of GCs impact across the rheumatic diseases.
Data availability statement
No data are available.
Ethics statements
Patient consent for publication
Ethics approval
Ethical approval was obtained from the University of the West of England, Bristol (UWE REC Ref: HAS.21.09.011). Participants gave informed consent to participate in the study before taking part.
Acknowledgments
The authors are grateful to the following organisations and patient groups, and to all others who shared the survey link on social media platforms: Rare Autoimmune Rheumatic Diseases Alliance (RAIRDA), Vasculitis UK, Lupus UK, Scleroderma and Raynaud’s UK, The British Sjögren’s Syndrome Association, Vasculitis UK, Lupus UK, Myositis UK, National Rheumatoid Arthritis Society (NRAS), Relapsing Polychondritis Foundation, Versus Arthritis, Vasculitis OZ, Arthritis Australia, Vasculitis Foundation, Outcome Measures in Rheumatology (OMERACT) Patient Research Partners, Hospital for Special Surgery lupus programs, North Georgia Rheumatology Group, UK Gout Society, Lupus and Allied Diseases Association Inc, Scotland Versus Arthritis, Northern Ireland Versus Arthritis, PMRGCAukFacebook groups: CreakyJoints patient advocates, Rheumatoid Arthritis Group, University Hospitals Bristol and Weston Foundation Trust Patient Action Group, SarcoidosisUK, Anti-inflammatory lifestyle, Vasculitis Support Forum, Rheumatoid Arthritis Support
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Josef S Smolen
Twitter @Sarah_L_Mackie, @Robsonjo4
SB and MN contributed equally.
Contributors SB: data curation, formal analysis, investigation, project administration, software, visualisation, writing—original draft, and writing—review and editing. MN: data curation, formal analysis, investigation, methodology, project administration, software, validation, visualisation, and writing—review and editing. JD and SMG: conceptualisation, methodology, validation, and writing—review and editing. PR and CS: conceptualisation, data curation, investigation, validation, visualisation, and writing—review and editing. ED: conceptualisation, investigation, methodology, validation, and writing—review and editing. CH: conceptualisation, investigation, methodology, validation, and writing—review and editing. SLM: conceptualisation, investigation, methodology, validation, and writing—review and editing. JR: conceptualisation, data curation, formal analysis, funding acquisition, investigation, methodology, project administration, resources, supervision, validation, visualisation, writing—original draft, and writing—review and editing. JR and MN accepts responsibility for the work and conduct of the study, had access to the data and controlled the decision to publish.
Funding This study was funded by Vifor Pharma (Unrestricted grant).
Competing interests SB—Grant/research support from: Vifor Pharma; MN—Grant /research support from: Vifor Pharma; JD, PR, CS and CH: None declared; ED—Grant/research support from: Vifor Pharma; SMG—Consultant of: UCB, Grant/research support from: Novartis; SLM reports: Consultancy on behalf of her institution for Roche/Chugai, Sanofi, AbbVie, AstraZeneca, Pfizer; Investigator on clinical trials for Sanofi, GSK, Sparrow; speaking/lecturing on behalf of her institution for Roche/Chugai, Vifor, Pfizer, UCB, Novartis and AbbVie; chief investigator on STERLING-PMR trial, funded by NIHR; patron of the charity PMRGCAuk. No personal remuneration was received for any of the above activities. Support from Roche/Chugai to attend EULAR2019 in person and from Pfizer to attend ACR Convergence 2021 virtually. SLM is supported in part by the NIHR Leeds Biomedical Research Centre. The views expressed in this article are those of the authors and not necessarily those of the NIHR, the NIHR Leeds Biomedical Research Centre, the National Health Service or the UK Department of Health and Social Care; JR—Speakers bureau: Vifor Pharma, Consultant of: Vifor Pharma, Unrestricted Grant/research support from: Vifor Pharma.
Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the Methods section for further details.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.