Article Text
PDF
Abstract
Objectives (1) To investigate the psychometric properties of a patient-reported numerical rating scale (NRS) for evaluating functional disability in osteoarthritis (OA), in comparison with the WOMAC function scale and with a physician-reported function NRS; (2) to estimate the patient acceptable symptomatic state (PASS) and the minimal clinically important improvement (MCII) values for treatment with non-steroidal anti-inflammatory drugs (NSAIDs).
Methods Data were extracted from a prospective multicentre study involving 1186 patients with knee or hip OA. The psychometric properties assessed were feasibility: percentage of responses, floor and ceiling effects; construct validity by examining the correlations with classically used OA outcomes measures; responsiveness by comparing the results of before and 1 month after treatment with NSAIDs using standardised response mean (SRM) and effect size (ES). The MCII and PASS values of each function scale were calculated by an anchoring method.
Results No floor or ceiling effect was observed. High correlations were observed as expected between the patient NRS and WOMAC function, pain visual analogue scale and patient global assessment. The responsiveness was moderate to large, with SRM and ES ranging from 0.6 (hip OA) to 0.9 (knee OA) and higher than that of the WOMAC function scale. The PASS was close to 3 for the NRS scales. The MCII appears to be the change that makes the OA functional disability decrease from baseline to the PASS.
Conclusion The patient-reported NRS demonstrated good psychometric properties, similar to the WOMAC function scale and can be regarded as a promising tool in therapeutic evaluation and decision-making in OA.
Statistics from Altmetric.com
Patient-reported outcomes (PROs) are of increasing interest in clinical practice and clinical research.1 2 Developing and validating such PROs has been emphasised over recent years in osteoarthritis (OA).3,–,5 In knee and hip OA, three core patient-reported domains related to the symptomatic severity should be systematically evaluated: pain, physical function and patient global assessment (PGA).6 7
Among the several disease-specific instruments used to assess functional impairment in OA, the Western Ontario and McMaster Universities osteoarthritis index (WOMAC) function subscale8 is the most widely used in clinical trials. However, its application for use in routine practice remains difficult, as the completion time is high for this 17-item questionnaire with 1–5 Likert scale responses. Moreover, some other disadvantages have been noticed, such as the different interpretations that can be attributed to the adjectives of the scale, the unequal intervals between the categories and the risk of mistakes when calculating the WOMAC function score.9 Conversely, the use of a visual analogue scale (VAS) or 0–10 numerical rating scale (NRS) is applicable in clinical practice for monitoring a chronic rheumatic disease10 and facilitates statistical analyses in clinical research. Therefore, a patient-reported function NRS could be helpful in OA, both in daily practice and clinical research, especially when completion of repeated tests is required. The use of an NRS scale for functional disability is advocated in rheumatoid arthritis11 12 but its usefulness has never been validated in OA. Another tool which has been widely used in clinical practice but rarely studied is an assessment of functional disability directly by the physician (physician-reported function NRS).
The question of what constitutes an important improvement or acceptable state in functional disability for patient is also a major issue, as results can then be easily expressed at the level of individual response, which facilitates their interpretation and transposition in clinical practice.13 Thus, the concepts of minimal clinically important improvement (MCII) and patient acceptable symptomatic state (PASS) have been developed to help in interpreting the results of trials involving PROs.13,–,15 In OA, MCII and PASS values have been estimated for the WOMAC function scale16 and for pain VAS.17 Therefore, the MCII and PASS values of each new PRO to be used in OA trials should be assessed.15 Thus, the objectives of this study were:
(a) To investigate the psychometric properties of a patient-reported NRS, in comparison with the WOMAC function scale and with a physician-reported NRS.
(b) To estimate the PASS and MCII values of these NRS scales for non-steroidal anti-inflammatory drug (NSAID) treatment.
Patients and methods
Study population
Data were extracted from a previously-reported prospective study (MOVE),16 involving outpatients with hip or knee OA, as defined by the American College of Rheumatology.18 19 Briefly, all patients were recruited by 399 French rheumatologists in private practice. To be included, patients had to experience pain related to OA >30 mm on a 0–100 VAS and to require treatment with NSAIDs. All patients initially visited their rheumatologist and inclusion began with the onset of NSAID treatment or with a switch from one NSAID to another. A final visit to the same rheumatologist was scheduled 4 weeks later. The study protocol was approved by the ethics committee of Cochin Hospital (Paris, France) and was conducted in compliance with good clinical practice and the Declaration of Helsinki principles.
Outcome measures
Function NRS
All patients were asked to assess their functional impairment on an 11-point NRS (patient NRS), the score ranging from 0 to 10; high scores indicate a high level of disability. The patient NRS wording was: “What is the degree of difficulty you have experienced for the daily activities during the last 48 hours due to your (knee or hip) OA” (online supplementary data). This PRO was assessed at baseline and after 4 weeks, without knowledge of the previous result.
For explanatory purposes, the physicians were also asked to estimate the functional impairment of each patient on an 11-point NRS (physician NRS) at baseline and final visit, without knowledge of the patients' answers.
Other measurements
At the baseline visit, demographic (age, gender, body mass index) and disease data (disease duration, radiological Kellgren and Lawrence grade,20 current symptomatic slow-acting OA drugs and NSAID intake) were collected.
At baseline and at the final visit, all patients were asked to assess the following PROs: pain on a 0–10 VAS; functional disability measured on the French version of the WOMAC function subscale (five-point Likert version) with total score linearly transformed to a 0–100 scale (higher scores indicate worse status)21; PGA of disease activity on a 0–10 VAS. Additionally, physicians assessed each patient's global disease activity on a 0–10 VAS at baseline and at final visit (physician global assessment).
PASS and MCII
The MCII was defined as the smallest change in measurement that signifies an important improvement in patient's symptoms.13 15 The PASS14 15 17 was defined as the value of the measurement beyond which patient consider themselves well. All patients had to assess:
▶ Their degree of improvement of global state (global MCII), on a three-point Likert scale (worsened function, no change, improved function). Among the patients who improved, the degree of improvement was scored on a four-point-Likert scale (poor, fair, good, excellent).16
▶ Their current global state (global PASS) by answering ‘Yes’ or ‘No’ in answer to the question ‘Taking into account all the activities you have during your daily life, your level of pain, and also your functional impairment, do you consider that your current state is satisfactory?’.17 The global PASS question was used as the ‘gold standard’ of patient's opinion for discriminant capacity.
For exploratory purposes, the MCII and PASS for functional status were also calculated. The global and function PASS values and global and function MCII values of each function scale were calculated at the final visit, as explained in the ‘Statistical analysis’ paragraph.
Statistical analysis
Analysis was conducted separately for patients with knee or hip OA. First, a descriptive analysis of disease data and demographic variables was performed and the difference between the mean score of patient and physician NRS was assessed by Wilcoxon test. The psychometric properties of the patient NRS were then evaluated and compared with those of the WOMAC function subscale (‘gold standard’ for OA functional disability assessment) and with those of the physician NRS.
Feasibility
Feasibility of the function NRS was assessed through the percentages of missing data and using the floor and ceiling effects in patients with hip or knee OA at baseline. In this study, floor and ceiling effects were considered present if more than 15% of the respondents achieved the highest or lowest score.22
Validity
Construct validity
The construct validity was assessed using Spearman's coefficient by examining the correlations between the results of the patient NRS with the other OA outcome measures (physician NRS, WOMAC function scale, pain VAS, PGA, physician global assessment). Correlation between NRS and OA radiographic severity was also determined using a non-parametric one-way analysis of variance (Kruskal–Wallis test). A priori hypotheses were that the patient NRS would be strongly (r>0.6) correlated with other function scales (WOMAC and physician NRS) and with other PROs (pain VAS and PGA). In addition, it was hypothesised that correlations would be higher between patient-reported scales than between patient versus physician-reported scales,23 and between single-item scales than between multi-item (WOMAC) versus single-item scales, whatever the construct.
Discriminant capacity
Discriminant capacity was defined as the ability of the function scales to classify patients achieving (ie, who answered ‘Yes’ to the PASS question) or not the PASS at the end of the study. It was evaluated for each function scale by comparing the mean scores between patients who achieved and patients who did not achieve the PASS, using the Wilcoxon test.
Responsiveness
Responsiveness of the function scale scores, of VAS pain and PGA was assessed by comparing before and 1 month after treatment with NSAIDs using the standardised response mean (SRM), defined as the mean change between baseline and follow-up divided by the SD of this change, and the effect size (ES), defined as the mean score change between baseline and follow-up divided by the SD of the baseline values. SRM and ES can be considered as large (>0.80), moderate (0.5–0.80) or small (<0.50).24 Calculation of 95% CI was performed by bootstrap. These measures were determined for patients who were NSAID-naïve or who switched NSAIDs.
MCII and PASS
The MCII of each function scale was defined as the 75th centile of the absolute change in score among patients whose final evaluation of response to a NSAID was improved (improvement good or excellent).16 The PASS of each function scale was defined as the 75th centile of the absolute score among patients who considered their final state as satisfactory.25
Statistical analyses were performed with the SAS 9.1 statistical software.
Results
In all, 881 patients with knee OA and 305 patients with hip OA were enrolled (table 1). Mean age of the patients was 66.7±11.1 years, 67.7% were female and mean OA duration was 4.1±5.4 years. Patients had high functional impairment (mean WOMAC function=46.2±16.4) without significant difference between the patient and physician NRS (5.93±1.92 vs 5.45±1.72 (p=0.56) for knee and 5.69±2.04 vs 5.41±1.77 (p=0.37) for hip, respectively). Physicians systematically underscored OA functional disability (NRS) and activity (global assessment) but these differences did not reach statistical significance. The distribution of NRS scores is shown in online supplementary figures.
Baseline characteristics of patients
Feasibility
At baseline, the missing data rates were 2.8% (n=33) and 3.1% (n=37) for the patient NRS and the physician NRS, respectively. In comparison, the percentage of missing data for the WOMAC function subscale was 8.1% (n=97). No floor or ceiling effect was observed in the study population for any of the function scales.
Validity
Construct validity
Clinical outcome measures (table 2): strong correlations were observed between the patient NRS and the OA classically used PROs (pain VAS, WOMAC function and PGA), as a priori hypothesised. The patient NRS was the function scale with the highest correlation with these three PROs, without significant difference between knee and hip. As expected, numerical scales were all more strongly correlated with the global assessment when the evaluation was performed only by the patient (patient NRS vs PGA, r=0.714 for knee OA) or only by the physician (physician NRS vs physician global assessment, r=0.785 for knee OA). In comparison with patient NRS, the WOMAC function was moderately correlated with VAS pain and PGA in knee and hip OA.
Construct validity of function scales determined by Spearman's correlations at baseline
Radiographic severity (online supplementary material, table A): symptomatic level was slightly higher for patients with greater radiographic severity, though this trend did not always reach statistical significance. In patients with knee OA, both patient and physician NRS appeared to be more discriminant for radiological severity than the WOMAC function subscale.
Discriminant capacity
All three function scales had a similar and good discriminant ability to distinguish between patients satisfied or not with their global state at the end of the study with similar effect sizes (2.84–3.04) (table 3).
Discriminant capacity of function scales in patients with hip and knee OA for global PASS achievement
Responsiveness
The responsiveness of the patient NRS after treatment with NSAIDs was moderate to large, with SRMs ranging from 0.73 (hip OA) to 0.83 (knee OA) and effect sizes ranging from 0.64 (hip OA) to 0.89 (knee OA) and similar to those of physician NRS (table 4). However, the WOMAC function subscale appeared to be less responsive than the numerical scales for both knee and hip, with SRMs ranging from 0.49 (hip OA) to 0.64 (knee OA). For comparison, the SRMs of PGA (0.66 and 0.83 for hip and knee, respectively) and VAS pain (0.70 and 0.86, respectively) were similar to those of NRS scales. As expected, NSAID-naïve patients experienced greater improvement in physical function after 4 weeks of treatment with NSAIDs than patients who were switching from one to another NSAID (supplementary material, table B).
Responsiveness of function scales in hip and knee OA
MCII/PASS
Using MCII/PASS questions focusing on functional impairment, 53.8% of patients with knee OA and 50.4% patients with hip OA indicated a functional improvement after treatment with NSAIDs and 67.8% and 65.9% considered their functional state at week 4 as satisfactory.
Patients with knee OA considered their global state as improved for a change of patient NRS ≥2.72 (global MCII) and their functional state as satisfactory if the patient NRS was <3.30 (function PASS) (table 5). Results for MCII and PASS were similar for patient and physician NRS. Conversely, the MCII values of the WOMAC function were proportionally smaller than the MCII values of the NRS, both for hip and knee OA. An interrelationship was observed between the MCII and the PASS for the three function scales (figure 1). When subtracting the MCII from the mean baseline value of the scale, the result obtained was very close to the PASS. In others words, the smallest change considered important to the patients (MCII) appears to be the change that makes the functional disability decrease from baseline to satisfactory state (PASS).
(A) Baseline mean scores, minimal clinically important improvement (MCII) and patient acceptable symptom state (PASS) in patients with knee osteoarthritis (OA) for function scales (patient NRS, physician NRS and WOMAC function). The inter-relationship is seen by subtracting the MCII from the baseline mean score for the three function scales; the value obtained is very close to the PASS. See ‘Results’ section for further detail. (B) Baseline mean scores, MCII and PASS in patients with hip OA for function scales (patient NRS, physician NRS and WOMAC function). The inter-relationship is seen by subtracting the MCII from the baseline mean score for the three function scales; the value obtained is very close to the PASS. See ‘Results’ section for further detail. NRS, numerical rating scale; WOMAC, Western Ontario and McMaster Universities osteoarthritis index.
Patient acceptable symptom state (PASS) and minimal clinically important improvement (MCII) scores for global state and functional state in patients with knee or hip OA
Discussion
This study which enrolled a large cohort of symptomatic patients with OA requiring treatment with NSAIDs validates a new, copyright-free instrument to assess functional impairment, the patient-reported NRS. The psychometric properties of this PRO were found to be satisfactory as a measure of feasibility, construct validity, responsiveness and discriminant capacity against the patient's perspective (PASS question). These results were similar to those of the WOMAC function scale; these latter were in accordance with previous studies.8 26,–,28 Thus, the patient NRS might be an alternative to the WOMAC function scale in OA trials, provided that the reliability of this PRO is confirmed.
Indeed, the test–retest reproducibility of the patient NRS not evaluated in this study needs to be assessed in further studies to ensure that its reliability is good and similar to that of the WOMAC function. Another limitation of this study might be the ability of a single-item scale to accurately assess the severity of functional impairments. Indeed, a multi-item questionnaire, such as WOMAC function, provides a more precise measure of functional impairment as each item is associated with a particular level of difficulty.12 However, when precise measurement of functional status may be less important, such as the measurement of changes in clinical trials or clinical practice, a patient NRS can be a valuable PRO.
The results of correlation with classically used OA outcome measures support the idea that the patient NRS shows evidence of convergent construct validity. As expected, high correlations occurred when comparing the patient NRS with other function scales, as they are intended to measure a similar construct. Analyses of function NRS, pain VAS and PGA confirmed that the three core domains are strongly related in patients with OA, as previously described.3 29 30 However, these correlations did not reach 0.72, indicating that pain, function and PGA domains are not totally overlapping in OA (in this study, patient NRS explained no more than 51% of the variance of the PGA (data not shown)). Furthermore, experts in the field of OA recommend examining the three domains separately.6 7 Correlation analyses suggest that the WOMAC function subscale might be more conceptually distinct from VAS pain and PGA than the function NRS, both in patients with knee and hip OA, perhaps reflecting the difficulty of capturing the complexity of functional disability in a single item.
Improving the responsiveness of a new evaluative tool is a key issue as it allows to the sample size to be reduced, which is of particular interest in OA clinical trials.31 The responsiveness of the patient NRS was moderate (knee OA) to good (hip OA) for a 4-week treatment with NSAIDs. SRMs and ES of the patient NRS were slightly higher than those of the WOMAC function for knee and hip OA and reached those of PGA or VAS pain. Moreover, the responsiveness of the patient NRS was also illustrated by the SRM analyses between NSAID-naïve and NSAID-experienced patients.
The use of MCII and PASS is of increasing interest in OA clinical research16 17 and in routine practice32 to define the thresholds for monitoring response to treatment.15 In this study, the values of MCII and PASS for the function NRS were obtained for the first time. As previously reported in OA for other PROs, the PASS is close to 3/10. The MCII of the WOMAC function was proportionally smaller than the MCII of the patient NRS as the MCII is dependent on the baseline score.32 As previously demonstrated for the WOMAC in OA,32 the MCII of the NRS seems to be the change needed to reach the PASS. This inter-relation between MCII and PASS may have implications for defining treatment success by intensifying the treatment until the patient achieves the PASS.
Although PROs have become used more and more often in the assessment of rheumatic patients in clinical trials,1 2 5 they remain underused in routine care10 33 as they are regarded as relatively cumbersome instruments.34 35 In this context, a single-item scale can be a practical help and would induce more clinicians to quantify OA outcomes in daily practice.3 Moreover, a WOMAC score may be difficult to interpret—in particular, in clinical practice.10 By contrast, the patient NRS can be scored instantaneously and intuitively interpreted. In clinical research, feasibility is less of a problem and time-consuming questionnaires are acceptable.36 However, even in this context, shortened tools have been developed in OA over the past 10 years25 37,–,39 and may be very useful, especially if multiple domains of health are assessed leading to voluminous case report forms. Moreover, the cross-cultural adaptation of a PRO which is a complex, time-consuming but indispensable process for use in international studies is facilitated for one or few questions.
In conclusion, this study indicates that the patient-reported function NRS is a promising tool in therapeutic evaluation and decision-making in OA clinical research and is practical for use in routine care. Further studies are needed to validate these preliminary encouraging results.
Acknowledgments
The authors thank all the rheumatologists who recruited patients for this study and Philip Bastable for a helpful review of this article.
References
Supplementary materials
Web Only Data
Files in this Data Supplement:
Footnotes
-
Funding This study was supported by an unrestricted grant from Merck, Sharp & Dohme Chibret Laboratories, France.
-
Competing interests None.
-
Patient consent Obtained.
-
Ethics approval This study was conducted with the approval of the Paris-Cochin ethics committee.
-
Provenance and peer review Not commissioned; externally peer reviewed.