Objectives: To validate the WOMAC 3.1 in a touch screen computer format, which applies each question as a cartoon in writing and in speech (QUALITOUCH method), and to assess patient acceptance of the computer touch screen version.
Methods: The paper and computer formats of WOMAC 3.1 were applied in random order to 53 subjects with hip or knee osteoarthritis. The mean age of the subjects was 64 years (range 45 to 83), 60% were male, 53% were 65 years or older, and 53% used computers at home or at work. Agreement between formats was assessed by intraclass correlation coefficients (ICCs). Preferences were assessed with a supplementary questionnaire.
Results: ICCs between formats were 0.92 (95% confidence interval, 0.87 to 0.96) for pain; 0.94 (0.90 to 0.97) for stiffness, and 0.96 (0.94 to 0.98) for function. ICCs were similar in men and women, in subjects with or without previous computer experience, and in subjects below or above age 65. The computer format was found easier to use by 26% of the subjects, the paper format by 8%, and 66% were undecided. Overall, 53% of subjects preferred the computer format, while 9% preferred the paper format, and 38% were undecided.
Conclusion: The computer format of the WOMAC 3.1 is a reliable assessment tool. Agreement between computer and paper formats was independent of computer experience, age, or sex. Thus the computer format may help improve patient follow up by meeting patients’ preferences and providing immediate results.
- ICC, intraclass correlation coefficient
- WOMAC, Western Ontario and McMaster Universities osteoarthritis index
- osteoarthritis index
Statistics from Altmetric.com
- ICC, intraclass correlation coefficient
- WOMAC, Western Ontario and McMaster Universities osteoarthritis index
The WOMAC (Western Ontario and McMaster Universities) osteoarthritis index is the best validated and most widely used outcome measure in subjects with hip or knee osteoarthritis.1,2 It is a 24 item questionnaire focusing on pain, stiffness, and functional limitation.3
The WOMAC index has been used as the main outcome in evaluation of pharmacological4 and surgical trials,5,6 as well as observational studies.7,8 Several studies have assessed its validity, reliability, and responsiveness.4,8,9,10 The recently defined OARSI (OsteoArthritis Research Society International) response criteria for clinical trials are based largely on the WOMAC index.11
The WOMAC index is usually administered in paper format. Only recently have computerised versions of the index been developed.12,13 The advantages of computerised applications are direct data entry—decreasing the chances of error in data transcription—and the possibility of an immediate display of results.12 In addition, the touch screen computerised format applies each question as a cartoon, in writing, and in speech, which may be appreciated particularly by elderly subjects. However, patients’ preferences and patients’ acceptance of the touch screen computerised format of the WOMAC index have not been evaluated.
In this study we aimed to validate the latest version of the instrument, WOMAC 3.1, in a Likert scale on a touch screen computer. In addition, we assessed patients’ preferences and acceptance of the touch screen computer format compared with the paper format, taking into account previous computer experience, age, and sex.
The study included 53 eligible subjects with symptomatic osteoarthritis of the hip or knee seen consecutively in our rheumatology outpatient clinic (Kantonal Hospital Aarau, Switzerland). Subjects underwent a lower extremity clinical examination for signs of hip and knee osteoarthritis. This included inspection, manual examination by palpation, and functional examination of range of motion (deficits in flexion, extension, and rotation; instability; joint laxity). To be enrolled in the study subjects had to have a positive clinical examination for knee or hip osteoarthritis as well as radiographic changes in the symptomatic joint consistent with osteoarthritis. In addition, subjects needed to be fluent in the German language. Subjects with symptomatic disease after total joint replacement were excluded. The characteristics of the patients are shown in table 1.
Subjects were invited to complete both the paper format and the touch screen computerised version of the WOMAC osteoarthritis index, version 3.1. They received either the paper format first or the touch screen computer format first, in random order. There was a 15 minute break between the two assessments. The German paper format of the WOMAC index used in this study has been validated previously.14
The computer touch screen format (the QUALITOUCH method12) presents each question of the WOMAC index as a cartoon, in writing, and in speech, on a 34.3 cm screen. The questions are answered by touching one of the five squares of the Likert scale on the screen. This may be done with a pen or by hand. Neither keyboard nor computer mouse is necessary. Difficulties are ranked as follows: 0, none; 1, mild; 2, moderate; 3, severe; 4, extreme. In addition to the Likert scale squares, there are four squares on the screen that the user could tap on: (1) help, (2) repeat, (3) back, (4) forward. With the forward option, it is possible to skip questions. The help function self activates if the screen is not touched for more than 15 seconds, guiding the user back to the question.
This data capturing method is called the QUALITOUCH method and has previously been validated for the WOMAC osteoarthritis index 3.0 numerical scale.12 The validation study of the WOMAC 3.0 against the paper format yielded the following intraclass correlation coefficients (ICCs) in patients with knee or hip osteoarthritis: pain, 0.91; stiffness, 0.74; function, 0.94. However, the WOMAC 3.0 validation did not include any measures of patients’ acceptance or previous computer use. In addition, no subgroup analyses by sex, age, and previous computer experience were carried out. Also, the WOMAC 3.1 uses a Likert scale, as oppose to the numerical rating scale used in the WOMAC 3.0 version. Overlapping both validation studies is the QUALITOUCH method, which presents each question in writing, in speech, and as a cartoon. The QUALITOUCH software for both versions of the WOMAC (3.0 and 3.1) is available through the authors.
WOMAC subscales scores were transformed to a 0–100 scale: a WOMAC score of 100 indicates that the patient has no problems and a score of 0 indicates that the patient has extreme difficulty. In between, a score of 25 indicates that a patient has severe difficulty, 50 indicates moderate difficulty, and 75 mild difficulty. Differences in WOMAC functional scores of more than 10 points on the transformed 0–100 WOMAC scale are generally perceptible to patients.15 Patients evaluated preoperatively before total joint replacement generally have WOMAC functional scores close to 50 or below.16 We compared patient classification (0, extreme difficulty; 1–24, very severe difficulty; 25–49, severe difficulty; 50–74, moderate difficulty; 75–99, mild difficulty; 100, no difficulty) between the paper and the touch screen computer formats for the function subscale.
After completing both formats of the WOMAC index, subjects were asked to fill in a short multiple choice supplementary questionnaire (five items are described in the results section and in fig 2 and table 3) on previous computer use and preferences with regard to the two formats.
Random assignment of which tool (paper or computer) was applied first was undertaken in blocks of 5. In the calculation of each of the three subscale scores of the WOMAC, the range of the subscale score was transformed to a range from 0 to 100 points, with a score of 100 indicating no pain or dysfunction:
100 – [actual raw score × 100/possible raw score range].
Descriptive statistics included the means and the standard deviation of the transformed WOMAC subscale scores. Paired t tests were used to compare differences between the paper and the computer version. In addition individual differences between formats are displayed graphically in fig 1. Individuals with differences of 10 points on the 0 to 100 scale between formats were considered discrepant on each subscale, following the suggestions of Ehrich and colleagues.15 Agreement (reliability) was assessed with the intra-class correlation coefficient. Ease of use and preference among subjects who expressed a preference were compared with a two tailed comparison test against 0.5.
Data were analysed using SAS version 8.1 and SPSS version 11.
Table 1 shows the characteristics of the study population. In all, 53 subjects with symptomatic knee or hip osteoarthritis were enrolled in the study. The age range was 45 to 83 years, with a mean of 64 years. Sixty per cent of participants were male.
Every participant completed both formats of the WOMAC index, version 3.1. There were 0.55% of items missing on the computer format (seven items from two persons) and 0.31% of items missing on the paper format (four items from two persons). The means and standard deviations of the WOMAC aggregate subscale scores are presented in table 2. For all subscales, aggregate means were similar between the paper and the computer versions. Based on the mean score difference by subscale, there was no significant difference between the paper and the computer version. Agreement as assessed by the intraclass correlation coefficient was very good for all three subscales: pain, 0.92; stiffness, 0.94; function, 0.96. If people with hip osteoarthritis were excluded, ICCs were similar to the group that included all persons: pain, 0.91; stiffness, 0.93; function, 0.96.
The difference between the two formats was plotted against the paper format as the gold standard in fig 1 to further illustrate the differences in the formats by subscale in individuals. There was no systematic error in any of the subscales. For all three subscales, the majority of subjects showed no or minimal differences between paper and computer formats, but when discrepancies occurred, they tended to be larger in the midrange of the scale. In all there were 10 persons who scored 10 points lower or higher in the computer format than in the paper format (three for pain, three for stiffness, and four for function). None of these 10 persons was discrepant in more than one subscale, and seven of the 10 were aged 65 years or older.
In the additional questionnaire on previous computer use and preferences, 53% of subjects stated that they used a computer previously either at work (28%) or at home (25%). Only 28% of all subjects used the internet.
If agreement between formats was assessed stratified by previous computer use, we found that the ICCs between the paper and the computer format were similar for subjects with and without computer experience. The ICCs for subjects with previous computer experience were as follows: pain subscale, 0.92 (95% confidence interval (CI), 0.82 to 0.96); stiffness subscale, 0.95 (0.88 to 0.98); and function subscale, 0.96 (0.91 to 0.98). The ICCs for subjects without previous computer experience were: pain subscale, 0. 92 (0.81 to 0.96); stiffness subscale, 0.94 (0.86 to 0.97); and function subscale, 0.97 (0.92 to 0.98).
Fifty three per cent of the study sample were 65 years or older. If agreement between formats was assessed stratified by age, we found that the ICCs between the paper and the computer format were similar for subjects below age 65 and those aged 65 years or older. The ICCs for subjects aged ⩽64 years were as follows: pain subscale, 0. 89 (0.76 to 0.95); stiffness subscale, 0.91 (0.79 to 0.96); and function subscale, 0.98 (0.95 to 0.99). The ICCs for subjects 65 years or older were: pain subscale, 0. 95 (0.88 to 0.98); stiffness subscale, 0.97 (0.93 to 0.98); and function subscale, 0.95 (0.89 to 0.98).
Sixty per cent of the study sample were men. If agreement between formats was assessed stratified by sex, we found that the ICCs between the paper and the computer format were similar for male and female subjects. The ICCs for men were as follows: pain subscale, 0.94 (0.88 to 0.97); stiffness subscale, 0.94 (0.88 to 0.97); and function subscale, 0.97 (0.94 to 0.99). The ICCs for women were: pain subscale, 0. 87 (0.67 to 0.95); stiffness subscale, 0.94 (0.85 to 0.97); and function subscale, 0.94 (0.85 to 0.98).
If we applied clinical cut off points for the aggregate function subscale, there were no subjects with extreme and very severe functional limitations (score of 0–24) in either format, while there were eight subjects with severe functional limitations (score of 25–49) in the paper format and five in the computer format. There were 22 subjects in the paper format and 24 in the computer format with moderate functional limitations (score of 50–74), and 23 subjects in the paper format and 24 in the computer format with mild functional limitations (75–99). None of the subjects had a function score of 100.
Figure 2 gives the results relating to two questions in the supplement questionnaire. For the item “ease of use,” subjects were asked the following question: “Which format was easier for you to use: the paper format, the computer format, or both equally?” For the item “preference,” subjects were asked the following question: “Which format did you prefer: the paper format, the computer format, or both equally?” We found that 66% of subjects stated that both formats were equally easy to use and 26% thought the computer format was easier to use. Only 8% of subjects felt that the paper format was easier to use. The majority of subjects (53%) preferred the computer format over the paper format (9%), and 38% of subjects had no preference. In subjects who expressed a preference, there were approximately three times as many who found the computer format easier to use (p = 0.02) and approximately five times as many who preferred the computer format over the paper format (p<0.01).
We asked subjects more specifically about three features of the two formats, as shown in table 3. Ninety four per cent of subjects felt that the combination of cartoon, writing, and voice of the computer format was informative and helpful, 2% were undecided, and 4% were irritated. Sixty per cent stated that it is informative and helpful that the computer format presents only one question at a time, while 40% had no preference. With regard to the paper format, 49% felt that it was informative and helpful that it allowed one to go forward and backward between pages, while 51% had no preference.
The touch screen computer format of the WOMAC osteoarthritis index was designed to facilitate patient assessment in clinical practice and research. In this study, we show that the touch screen computer format of the WOMAC index version 3.1 is a reliable assessment tool in patients with hip or knee osteoarthritis compared with the original paper format, independent of previous computer experience, age, and sex.
The ICCs for all three subscales (pain, stiffness, and function) showed very good agreement for the overall study sample, but also in subgroups with or without previous computer experience, and in subjects below age 65 or aged 65 years and older, and in men and women. A small percentage of subjects had scores with discrepancies of more than 10 points on the 0–100 scale in each subscale (three to four subjects of 53), the paper format being considered the gold standard. However, these subjects were only discrepant in one subscale, and the mean aggregate scores by subscale differed minimally between formats (0.11 to 1.5 points on the 0–100 scale). Most subjects with discrepancies of more than 10 points were 65 years and older. However, in subjects with discrepant scores, we could not determine whether the audiovisual presentation provided a score that was further from or closer to the “true” answer.
There were a few missing items for the computer format, which may reflect that fact that the software permitted questions to be left unanswered. While this is an important feature of personal choice, it may offer a chance of unintentionally missing a question. As suggested by Buxton et al, unintentional non-response may be avoided by having the skipped items presented a second time.17
If clinical cut off points for functional status were applied to both formats, there was very good agreement in levels of functional difficulty. Both formats classified the same subjects as having severe difficulties, and differed only in one person for moderate and mild difficulties.
In addition to the assessment of agreement between the paper and the touch screen computer format, we were interested in the patients’ acceptance and preferences. These were assessed by a supplementary five item questionnaire. Half our study sample was 65 years or older, 47% had never used a computer before, and only 15% used the internet. Nevertheless, 92% of the subjects either stated that both formats were equally easy to use or that the computer format was easier. Of the subjects expressing a preference, approximately three times as many found the computer format easier to use, and approximately five times as many preferred the computer format over the paper format. Most subjects appreciated the combination of cartoon, writing, and voice offered by the touch screen computer format (94%), and the majority stated that the presentation of a single question by the computer format was helpful and informative (60%).
In summary, there are several advantages of the touch screen computer format of the WOMAC osteoarthritis index (version 3.1). First, the computer format allows direct data entry and immediate display of results, which may improve patient monitoring in research and clinical practice. Second, we show that the computer format is reliable across subgroups of patients, including those without computer experience, the elderly, and both sexes. Third, the majority of subjects in this study found the computer format easier to use or as easy to use as the paper format, and among those who expressed a preference approximately five times as many preferred the computer format over the paper format. This suggests that the computer format finds acceptance among osteoarthritis patients evaluated in the outpatient clinic, and is preferred over the paper format by the majority of these patients.
We conclude that the touch screen format of the WOMAC osteoarthritis index 3.1 is a reliable assessment tool in persons with osteoarthritis at the hip or knee, independent of previous computer experience, age, and sex. This format may facilitate patient assessment in clinical practice and research.
We acknowledge the support of the department of orthopaedic surgery at the Kantonsspital Aarau, especially Dr Juerg Huber. We are indebted to Dr E John Orav (Department of Biostatistics, Harvard School of Public Health, Boston, Massachusetts, USA) for his statistical advice and Stephen M Ferrari (independent) for his insight and assistance in the preparation of this manuscript.