Assessing the reliability and responsiveness of 5 shoulder questionnaires*,**,*
Abstract
The purpose of this study was to compare the test-retest reliability and responsiveness of 5 different shoulder questionnaires in a sample of patients with shoulder pain. Ninety-nine patients completed the following shoulder questionnaires on 2 occasions: Subjective Shoulder Rating Scale, Simple Shoulder Test, Modified-American Shoulder and Elbow Surgeons Form, Shoulder Severity Index, and the Shoulder Pain and Disability Index. The Short Form-36 was also included. Test-retest reliability was assessed with intraclass correlation coefficients. Standardized response means were calculated to assess responsiveness. This procedure was done on 33 subjects who underwent rotator cuff surgery or total shoulder arthroplasty and who believed that they had improved between testings. All questionnaires had acceptable reliability (coefficients >0.75) except the Subjective Shoulder Rating Scale (coefficient = 0.71) and were more responsive (0.65 < Standardized response mean < 1.23) than the Short Form-36 (0.08 < Standardized response mean < 0.43) except for pain (0.91). In this longitudinal study a direct comparison of 5 shoulder questionnaires was carried out. We found the Subjective Shoulder Rating Scale to have lower reliability and responsiveness. The other 4 questionnaires including 1 developed to measure whole extremity function (Modified American Shoulder and Elbow Surgeons Form) rather than the shoulder only were comparable with good reliability and responsiveness. The results indicate that the shoulder questionnaires were more sensitive to change in patients with shoulder pain than the generic questionnaire (Short Form-36), and both types of questionnaires should be used in outcome evaluations. (J Shoulder Elbow Surg 1998;7:565-72)
References (0)
Cited by (258)
Correlation of patient-reported outcome measurement information system-upper extremity with American Shoulder and Elbow Surgeons scores in shoulder arthroplasty
2022, Seminars in Arthroplasty JSESMultiple outcome measure surveys are available to assess preoperative and postoperative outcomes for a variety of orthopedic procedures such as shoulder arthroplasty. Although legacy instruments such as American Shoulder and Elbow Surgeons (ASES) remain popular, there remains no singular gold standard survey instrument for shoulder arthroplasty patients, and alternative instruments have been developed to better capture clinical outcomes. The goal of this study is to compare the efficacy of Patient-Reported Outcome Measurement Information System-Upper Extremity (PROMIS-UE) using computer adaptive technology with ASES scores both preoperatively and postoperatively in shoulder arthroplasty patients. Our hypothesis is that there would be a strong correlation between PROMIS-UE and ASES scores both preoperatively and postoperatively in total shoulder arthroplasty patients.
Patients who underwent total shoulder arthroplasty and agreed to complete baseline and 1-year follow-up of ASES and PROMIS-UE scores were included. Patients also completed PROMIS-Physical Function (PROMIS-PF) as a third point comparison. Responses to these instruments were statistically analyzed and compared using Pearson correlation coefficients. Floor and ceiling effects were then calculated.
Ninety patients were included in this study, all of whom completed the PROMIS-UE, PROMIS-PF, and ASES surveys both preoperatively and 12 months postoperatively. The mean age in this cohort was 68.9 years (standard deviation [SD] 8.4 years, range 39-89). The mean preoperative and postoperative PROMIS-UE scores were 26.6 (SD 6.7, range 14.7-44.6) and 41.8 (SD 10.3, range 20.2-56.4) respectively. Mean preoperative and postoperative PROMIS-PF computer adaptive technology scores were 35.7 (SD 9.54, range 34.2-64) and 44.3 (SD 9.22, range 23.5-73.3) respectively. The mean ASES score was 37.5 preoperatively (SD 18.0, range 5-99.5) and 77.5 postoperatively (SD 20.5, range 25.0-100.0). PROMIS-UE demonstrated a moderate correlation with ASES preoperatively and a strong correlation postoperatively (r = 0.52, confidence interval 0.27-0.60; r = 0.70, confidence interval 0.55-0.78, respectively). PROMIS-UE demonstrated a minor floor effect preoperatively (7.8%) but significant ceiling effect postoperatively (24.4%) and ASES demonstrated a mild ceiling effect at final follow-up (8.9%). There were otherwise no other floor or ceiling effects at all other time points across each survey.
PROMIS-UE correlates well with ASES at both baseline and 1-year postoperation for patients undergoing total shoulder replacement. In addition, the change seen between both baseline and 1 year outcome scores for both scoring systems also correlates strongly, suggesting that PROMIS-UE may be a suitable alternative to ASES for this patient population.
Outcome Assessment Performance of the SF-36, Manchester-Oxford Foot Questionnaire and AOFAS in Forefoot Reconstruction Surgery
2022, Journal of Foot and Ankle SurgeryThe purpose of this study was to investigate the measurement properties of the Short Form 36 (SF-36) to detect real change after forefoot reconstruction surgery. Responsiveness and minimally important change estimates were compared with those from the Manchester-Oxford Foot Questionnaire (MOXFQ) and the American Orthopaedic Foot and Ankle Society (AOFAS) measures. Eighty-three patients awaiting surgery were recruited. Patients completed pre- and 12 months postoperative the SF-36 and the MOXFQ. A surgeon assessed the AOFAS scores. The responsiveness to change was determined using the effect size (ES), the minimal detectable change (MDC) and the minimal clinically important change. Two subscales of the SF-36 demonstrated significant improvement, bodily pain (BP) and mental health. Only the BP domain appeared the most responsive with an ES of 0.73. All domains of the MOXFQ and AOFAS produced much larger effect sizes (ES > 1.5). MDC values for the majority of the SF-36 domains fell within measurement error except for the BP domain. Fewer patients showed significant improvement when compared with the MOXFQ pain domain. In conclusion, the SF-36 measuring tool proved to be neither reliable nor responsive enough to detect real change after forefoot surgery. Though the BP domain appeared to be the most responsive, it failed to detect meaningful change when compared to the MOXFQ-Pain and the Visual Analogue Scale.
Significant Improvement in Shoulder Function and Pain in Patients Following Biologic Augmentation of Revision Arthroscopic Rotator Cuff Repair Using an Autologous Fibrin Scaffold and Bone Marrow Aspirate Derived From the Proximal Humerus
2021, Arthroscopy, Sports Medicine, and RehabilitationCitation Excerpt :Objective and subjective outcome measures including the American Shoulder and Elbow Surgeons (ASES) Score,17 Constant-Murley score,18 single assessment numeric evaluation score,19 Simple Shoulder Test,20 the visual analog scale for pain, and range of motion in terms of active flexion, abduction, and external rotation were collected preoperatively and at terminal follow-up with a minimum follow-up of 1 year (A.D.M.) Pain scores were obtained from the ASES survey. Previous studies have confirmed these scores in terms of reliability, validity, and responsiveness.19,21-23 In addition, the BMA of each patient was assessed for the number of nucleated cells and CFUs.
To clinically evaluate patients who underwent a biologic augmentation technique in revision arthroscopic rotator cuff repair using an autologous fibrin scaffold and concentrated stem cells isolated from bone marrow aspirate (BMA) obtained from the proximal humerus.
This is a retrospective review of prospectively collected data from patients who underwent biologic augmentation of revision arthroscopic rotator cuff repair using an autologous fibrin scaffold and BMA obtained from the proximal humerus between 2014 and 2015. Minimum follow-up was 12 months. Outcome measures were collected preoperatively and postoperatively including range of motion as well as American Shoulder and Elbow Surgeons Shoulder Form, Simple Shoulder Test, single assessment numeric evaluation, and visual analog score. In addition, BMA samples of each patient were assessed for the number of nucleated cells and colony-forming units. Regression analysis was performed to investigate whether the number of nucleated cells and colony-forming units had an influence on outcome and failure.
Ten patients who underwent biologic augmentation of revision arthroscopic rotator cuff repair using an autologous fibrin scaffold and concentrated BMA obtained from the proximal humerus between 2014 and 2015 were included. The mean follow-up time was 30.7 (range: 12-49) months. Four patients were revised at final follow-up. Postoperative clinical scores improved significantly: American Shoulder and Elbow Surgeons (28.1 ± 5.4 to 60.9 ± 9.0; P < .01), single assessment numeric evaluation (6.6 ± 2.3 to 65.1 ± 10.9; P < .01), visual analog scale (7.2 ± 0.9 to 3.1 ± 0.9; P < .01), and Simple Shoulder Test (1.6 ± 0.5 to 10.3 ± 5.7; P < .01). Postoperative range of motion increased significantly with regard to flexion (97.0 ± 13.6 to 151.0 ± 12.2; P < .01) and abduction (88.0 ± 14.0 to 134.0 ± 15.1; P = .038) but not with external rotation (38.0 ± 5.7 to 50.5 ± 6.5; P = .16). Less pain was correlated to an increased number of nucleated cells (P = .026); however, there was no correlation between failure rate and number of nucleated cells (P = .430).
Patients who underwent biologic augmentation of revision arthroscopic rotator cuff repair using an autologous fibrin scaffold and concentrated BMA demonstrated a significant improvement in shoulder function along with reduction of pain. However, the overall revision rate for this procedure was 40%.
Level IV, therapeutic case series.
Atraumatic shoulder instability: patient characteristics, comorbidities, and disability
2021, JSES InternationalAtraumatic shoulder instability of the shoulder is well described, and numerous extrinsic factors have been identified as having a negative influence on long-term shoulder function, including age, sex, smoking, and workers' compensation status. Furthermore, physical factors such as acute or chronic pain, as well as psychological comorbidities such as depression and suicidal thoughts, are common in shoulder-related conditions.
Patients with atraumatic shoulder instability were recruited from a review of outpatient logbook and inpatient rehabilitation center admission records. Average and worst pain visual analogue scale rates were recorded for acute and chronic shoulder pain. Disability was measured using the “Disability of the Arm and Hand” score, as well as the Stanmore Percentage of Normal Shoulder Assessment (SPONSA). Depression was assessed using Becks Depression Inventory II, and further single psychological items were selected from other psychological assessment questionnaires.
We included 64 patients, of which 51 (79.7%) were female. Mean visual analogue scale pain scores were in average 5.11 (standard deviation 2.24) and 5.58 (standard deviation 2.34) for acute and chronic shoulder pain, respectively. A stepwise, multiple linear regression revealed that only chronic shoulder pain remained significantly related to disability (F(1,61) = 46.13, P < .001). A second linear regression analysis was performed and showed a significant association between the overall Disability of the Arm, Shoulder and Hand score and overall Becks Depression Inventory score (F(1,62) = 12.78, P < .001). A further stepwise, multiple linear regression analysis indicated that chronic pain and depression remain independent variables in the prediction of disability.
In addition to sociodemographic factors, pain and psychological comorbidities were found to have a negative impact on patient's functional outcome. This study further supports the need for an multidisciplinary team, holistic approach in the management of atraumatic shoulder instability patients with particular emphasis on chronic pain management and psychological support.
Validation of a machine learning–derived clinical metric to quantify outcomes after total shoulder arthroplasty
2021, Journal of Shoulder and Elbow SurgeryWe propose a new clinical assessment tool constructed using machine learning, called the Shoulder Arthroplasty Smart (SAS) score to quantify outcomes following total shoulder arthroplasty (TSA).
Clinical data from 3667 TSA patients with 8104 postoperative follow-up reports were used to quantify the psychometric properties of validity, responsiveness, and clinical interpretability for the proposed SAS score and each of the Simple Shoulder Test (SST), Constant, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form (ASES), University of California Los Angeles (UCLA), and Shoulder Pain and Disability Index (SPADI) scores.
Convergent construct validity was demonstrated, with all 6 outcome measures being moderately to highly correlated preoperatively and highly correlated postoperatively when quantifying TSA outcomes. The SAS score was most correlated with the UCLA score and least correlated with the SST. No clinical outcome score exhibited significant floor effects preoperatively or postoperatively or significant ceiling effects preoperatively; however, significant ceiling effects occurred postoperatively for each of the SST (44.3%), UCLA (13.9%), ASES (18.7%), and SPADI (19.3%) measures. Ceiling effects were more pronounced for anatomic than reverse TSA, and generally, men, younger patients, and whites who received TSA were more likely to experience a ceiling effect than TSA patients who were female, older, and of non-white race or ethnicity. The SAS score had the least number of patients with floor and ceiling effects and also exhibited no response bias in any patient characteristic analyzed in this study. Regarding clinical interpretability, patient satisfaction anchor-based thresholds for minimal clinically importance difference and substantial clinical benefit were quantified for all 6 outcome measures; the SAS score thresholds were most similar in magnitude to the Constant score. Regarding responsiveness, all 6 outcome measures detected a large effect, with the UCLA exhibiting the most responsiveness and the SST exhibiting the least. Finally, each of the SAS, ASES, Constant, and SPADI scores had similarly large standardized response mean and effect size responsiveness.
The 6-question SAS score is an efficient TSA-specific outcome measure with equivalent or better validity, responsiveness, and clinical interpretability as 5 other historical assessment tools. The SAS score has an appropriate response range without floor or ceiling effects and without bias in any target patient characteristic, unlike the age, gender, or race/ethnicity bias observed in the ceiling scores with the other outcome measures. Because of these substantial benefits, we recommend the use of the new SAS score for quantifying TSA outcomes.
Rasch analysis of The Shoulder Pain and Disability Index (SPADI) in a postrepair rotator cuff sample
2021, Journal of Hand TherapyClinical measurement study: Level of evidence (N/A)
The Shoulder Pain and Disability Index (SPADI) is a self-reported outcome measure of pain and disability related to shoulder pathology. In comparison to Classical Test Theory (CTT), Rasch analysis offers a more rigorous examination of the measurement properties of a scale.
This study utilizes Rasch analysis to evaluate the psychometric properties of the SPADI to propose potential modifications and avenues for future investigation.
SPADI scores (n = 212) from participants one-year post rotator cuff repair were collected from an outpatient specialty clinic. Fit to the Rasch model, unidimensionality of the subscales, and areas of bias were evaluated.
Both the pain and disability subscales satisfied the requirements of the Rasch model with very minimal modifications and demonstrated unidimensionality. The person separation index was found to be high (P > .80), indicating reliability and internal consistency. Sex and the affected dominant side influenced how people scored on the SPADI (Differential item functioning (DIF)).
The findings suggest some patients in our sample have difficulty discriminating between item responses, particularly within the middle of the scale. Rasch analysis supports the clinical measurement properties of consistency and reliability, previously determined by CTT methods.
- *
Supported by a grant from the St Michael's Hospital Health Sciences Centre.
- **
Reprint requests: Robin Richards, MD, FRCS(C), 55 Queen St E, Ste 800, Toronto, ON, Canada, M5C 1R6.
- *
32/1/89820