Article Text
PDF
Abstract
Objectives To develop a questionnaire assessing the expectations of patients regarding management of osteoarthritis (OA of the knee.
Methods A detailed document reporting on a qualitative analysis of interviews of patients with knee OA was sent to experts and a Delphi procedure was adopted for item generation. Eighty physicians (64 general practitioners, 16 rheumatologists) recruited 566 patients with knee OA to test the provisional questionnaire. Items were reduced according to their metric properties and exploratory factor analysis. The reliability of the questionnaire was tested by the Cronbach α coefficient. Construct validity was tested by divergent validity and confirmatory factor analysis. Test–retest reliability was assessed by the intraclass correlation coefficient (ICC) and the Bland–Altman technique.
Results Sixty items were extracted from analysis of the interview data. The experts needed three Delphi rounds to obtain consensus on a 33-item provisional questionnaire. The item reduction process resulted in an 18-item questionnaire. Exploratory factor analysis extracted three main factors: factor 1 represented expectations for education, factor 2 expectations for information on technical and human support, and factor 3 expectations for physician empathy. The Cronbach α coefficient was 0.91 (95% CI 0.89 to 0.92). Expected divergent validity was observed. Confirmation factor analyses confirmed higher intra-factor than inter-factor correlations. Test–retest reliability was good with an ICC of 0.79, and Bland–Altman analysis did not reveal a systematic trend.
Conclusions A new 18-item questionnaire assessing patient expectations of management of knee OA by their physicians is proposed. The questionnaire has good content and construct validity.
Statistics from Altmetric.com
Introduction
Besides pain, disability and health-related quality of life are the main factors assessed in osteoarthritis (OA) and are usually measured by fixed-item questionnaires that do not take into account patient priorities. Patients with hip and knee OA or rheumatoid arthritis, healthy professionals and healthy controls do not agree on the importance of disabilities,1 ,2 and the views of patients and physicians on knee OA and its management differ.2 These discrepancies between patients and physicians in defining the importance of an illness should lead to a paradigm shift towards a more patient-centred approach. Taking into account concepts such as expectations in addition to disability and health-related quality of life may help to provide a better understanding of what is important to patients and to increase adherence to and efficacy of management strategies.3
The concept of expectation is complex and evolves with time and situation. For knee OA, patient expectations have been mainly studied with total joint replacement (TJR).4,–,7 Patient expectations are crucial for surgical decision-making, and the assessment of expectations regarding the functional benefits of TJR is recommended.8 Patient expectations about TJR depend on demographic, clinical, psychological and socioeconomic factors.9,–,12 In addition, ethnic origin, employment status and trust in a surgeon have a significant effect on expectations about TJR.9 Patients with more severe disease have higher expectations than those with less severe disease.11 Expectations regarding TJR can also vary between countries.13
Although patients with OA and their physicians differ in their assessment of what is important in terms of health and symptom status14 ,15 and surgical outcome,9 ,16,–,18 the expectations of patients concerning the management of knee OA have seldom been studied. Analysing patient expectations may help clinicians to understand better what is important for their patients and therefore enhance the patient–physician relationship and patient adherence to treatment. Measuring changes in or fulfilment of expectations and their impact on patient satisfaction and clinical outcomes may help researchers to optimise treatment strategies.
Qualitative research is probably the best way to understand patient needs and contexts and could improve therapeutic strategies and their assessments.19 The US Food and Drug Administration (FDA) has recently proposed guidelines for patient-reported outcomes which emphasise the need for semi-structured interviews of patients to ensure content validity of these instruments.20 We previously conducted a qualitative interview study exploring patient and practitioner views about knee OA and its management.21 This qualitative database may help in developing educational programmes on knee OA for patients and physicians and new patient-centred outcome measures assessing concepts such as expectations.
We aimed to use the data from this qualitative interview study to develop a questionnaire, the Knee Osteoarthritis Expectations Questionnaire (KOEQ), to assess patient expectations for the management of knee OA.
Methods
Development of the provisional questionnaire
We adopted a Delphi procedure to select items for the provisional questionnaire using previously described general methods for instrument development.22 The process consisted of three main steps: definition of the aim of the questionnaire, generation of items and selection of items.
Aim of the questionnaire
The general purpose of the questionnaire is to facilitate the patient–physician relationship and patient education by recording patient expectations in routine practice and clinical research. The specific purpose of this questionnaire is to better define what is important to patients, help to plan the management of their condition and to identify unrealistic expectations and try to modulate them.
Generation of items
The Delphi consensus method was used to generate and select items,22 with the initial development in French. For extracting items related to expectations, a detailed document reporting on the qualitative analysis of interviews with patients was sent to 10 experts (two general practitioners (GPs), two rheumatologists, two orthopaedic surgeons, one physical therapist, two sociologists and one physical and rehabilitation medicine physician). Experts were asked to read the documents and extract the most relevant items concerning patient expectations. To help experts, several domains were proposed: information and education (about illness, treatment options, prognosis, prevention, etc); access to healthcare (physicians, healthcare providers, biological and radiological explorations, etc); prescriptions (pharmacological, non-pharmacological, efficacy, side effects, etc); support (listening, availability of relatives, consideration of individual and material dimensions); and environment (adaptive devices, indoor and outdoor adaptations, human assistance). Experts were invited to add domains if they wished.
Selection of items
For each generated item, experts were asked to rate on two 11-point Likert scales from 0 (disagree) to 10 (agree) whether they believed the item should be selected in the final tool and the degree of agreement with the formulation of the item. Experts who disagreed with the formulation of the item were asked to propose a new formulation. Experts were also invited to add items to domains. Items with a median relevance score of ≤7 were excluded, as were redundant items.
For the second Delphi round, experts were asked to re-rank their agreement with each item; they could change their score in view of the group's response to the previous round but could not conform to the group's median response. A rewording of some items was proposed. Further, an explanation for the questionnaire and the modalities of answers were developed. Items with a median relevance score of ≤8 were excluded. A rewording of some items was proposed.
During the third and last round, experts had to comment on the final checklist and modalities of answers. Agreement was obtained with the third Delphi round.
Analysis of data
The responses for each Delphi round were reported as the percentage of experts choosing each value of the 11-point Likert scale. Experts' comments on each item were recorded. After each round the steering group (IB, SA, SP) discussed experts' qualitative and quantitative answers. From these answers, redundant items were combined, categories of items with insufficient consensus rates were excluded, items proposed by experts were added and items were modified or expanded.
English translation of the questionnaire
To provide a version of the questionnaire for English-speaking patients, the French version was translated by the forward and backward translation procedure.23 ,24 Two independent bilingual translators, whose native language was English, translated the French version of the questionnaire into English. As recommended, the translators were encouraged to strive for idiomatic rather than word-for-word translation. Two bilingual investigators (SP, FR) compared the two translated versions, with consensus. Two other independent translators who had not participated in the first stage and whose native language was French then back-translated the English version of the questionnaire into French. The investigators (SP, FR) then compared the translated version, with consensus.
Reduction of items and validation of the questionnaire
The aim was to select items with the best metric qualities from the provisional questionnaire and to assess the reliability and construct validity of the final questionnaire. We therefore conducted a national multicentre cross-sectional survey of patients in a primary care setting.
Recruitment of physicians and patients
Physicians
Rheumatologists and GPs were randomly selected from two national databases of 475 and 68 594 practitioners, respectively, who had not previously refused to participate in studies or surveys. The assigned physicians were contacted by mail, then telephone calls if they did not respond. The randomisation was stratified by geographical area. Each physician was asked to include five patients.
Patients
Each patient consulting one of the participating physicians for knee OA during the period of inclusion and meeting the inclusion criteria was asked to participate in the study. The inclusion criteria were age ≥45 years; knee OA defined by the American College of Rheumatology criteria,25 whatever knee OA activity status or treatment used; and written consent to participate in the study. The exclusion criteria were absence of knee x-rays and inability to complete a questionnaire. Patients were included from September 2009 to March 2010.
Data collection
Data recorded were patient characteristics: sociodemographic (age, sex, marital status, level of education (low, primary school; intermediate, secondary school; high, university or postgraduate), employment status, living area), medical (body mass index, duration of disease, comorbidities, type of knee OA (medial femorotibial, lateral femorotibial, femoropatellar)), level of physical activity, pharmacological and physical treatments for OA, and OA activity and function (pain intensity on an 11-point numerical scale (0–10), physician opinion of severity on an 11-point numeric scale (0–10), Western Ontario and McMaster Universities Arthritis Index (WOMAC) functional scale and SF-12 physical and mental scales). The questionnaires were completed on paper or electronically according to patient preference.
Sample size calculation
We expected a Cronbach α coefficient of 0.7–0.9 for the KOEQ. We needed to include 400 patients for a coefficient of 0.7 with 0.05 accuracy and one of 0.9 with 0.015 accuracy. This number of 400 patients was also sufficient for excellent accuracy of the coefficients on factorial analysis.
We assumed that each physician would include five patients and therefore planned to enrol 80 physicians (64 (80%) GPs and 16 (20%) rheumatologists). We hypothesised that 25% of the physicians would agree to participate and that 75% of these would include patients.
Statistical analysis
We used descriptive statistics to examine the response distribution to each item. Items with the following characteristics were removed: low response rate (≤95%); floor or ceiling effect, defined as >50% of the respondents choosing an extreme positive or negative response category, respectively; and high inter-item correlation (>0.70) assessed by Spearman correlation coefficient.
We used explanatory factor analyses with principal component analysis (PCA) to examine the construct validity of the KOEQ. Oblique promax rotation was selected because the factors were not expected to be completely independent of each other.26 Factors generated by PCA were extracted if eigenvalues were greater than the randomly generated factors from Horn's parallel analysis.27 Items were included in the factors if they revealed loadings greater than 0.5. In the case of multiple loading of an item on several factors, the item was included in the factor with a better conceptual relationship.
The internal consistency of the KOEQ was assessed by the Cronbach α coefficient to examine the degree to which the items in a scale measured the same concept28; a Cronbach α of >0.70 was considered acceptable, 0.71–0.80 respectable and >0.8 very good. The 95% CI of the Cronbach α was assessed by the bootstrap technique with 1000 replications.
The multi-trait method was used to test the significantly higher correlation of each item with items of its hypothesised factor than with items of the other factors.29 Distributions of intra-factor and inter-factor correlations were compared by a boxplot graph.
Divergent validity was assessed by Spearman correlation of the global score of the KOEQ and other outcome measures (knee OA severity assessed by physicians, knee pain, function WOMAC score and SF-12 physical and mental scores).
Test–retest reliability was assessed by the intraclass correlation coefficient (ICC) using a two-way random effects model (an ICC ≥0.75 is considered excellent reproducibility)30 and the Bland–Altman method by calculating the mean difference (δ) between two measurements and the SD of the difference.31 The 95% limits of agreement were defined as the mean difference between the measurements ±1.96 SD of the differences. By definition, if differences are normally distributed, 95% of individual differences are within 2 SD of the mean difference (ie, within the limits of agreement). The Bland–Altman plot is useful to search for any systematic bias, assess random error and reveal whether the difference between scores depends on the level of scores. We computed the SEM (essentially, the average SD among observations from the same subject).32 The SEM was estimated by calculating the square root of the within-subject variance (SEM=√σbetween measurement + σresidual). The smallest detectable change (SDC) was calculated by the formula SDC=1.96 × √2 × SEM. The SDC allows 95% confidence that the observed change is a real change.33 Patients from the cross-sectional survey could not participate in this part of the study because a visit to the physician might modify expectations. We therefore selected a sample of 40 patients from the files of the physical and rehabilitation medicine department and mailed them a questionnaire to complete at 2-week intervals.
The data were analysed with R 2.10.1 and SAS 9.1.
Results
Development of the questionnaire assessing patient expectations for management of knee OA
The experts did not generate new domains for the questionnaire. Synthesis of experts' responses to the analysis of the qualitative study led to the extraction of 60 items concerning expectations (figure 1).
Development of the Knee Osteoarthritis Expectation Questionnaire.
Selection of items
For the first Delphi round the 60-item questionnaire was mailed to the experts. The experts' responses were synthesised, items with median relevance ≤7 were eliminated and redundant items were combined, resulting in a 45-item questionnaire; 16 items were reformulated. Domains were combined into two categories: expectations from physicians and expectations from treatment.
For the second Delphi round the 45-item questionnaire was sent back to the experts along with the median scores for relevance and quality of the formulation of each item obtained during the first round, with minimum and maximum scores. The experts were asked again to rate the relevance and quality of the formulation of each item on two 11-point scales. After synthesis of the experts' responses a 33-item questionnaire assessing patient expectations for knee OA management was sent to experts for the third Delphi round (see table S1 in online supplement). Consensus was obtained at this stage.
English translation of the questionnaire
We identified a few discrepancies between each translation by the forward and backward translation procedure, but no information was distorted and consensus was easily obtained. Thus, the translated versions and the original versions explored the same dimensions.
Reduction of items and validation of the questionnaire
Patients
The physicians recruited 586 patients to test the questionnaire (figure 2). Five patients were excluded (four because the physician questionnaire could not be retrieved for verification of inclusion and exclusion criteria and one because he was aged 41 years), 37 patients did not return their questionnaire, leaving data from 524 patients for analysis.
Flow chart of practitioners through the trial.
The mean (SD) age of patients was 68.2 (10.1) years, disease duration 6.6 (5.3) years, pain intensity 5.6 (2.1 on a scale of 0–10) and the WOMAC score was 31.8 (12.9), range 1–62 (table 1).
Characteristics of patients with osteoarthritis (OA) of the knee surveyed in developing the Knee Osteoarthritis Expectation Questionnaire
The 38 patients not included in the validation of the questionnaire were on average 5 years younger), had shorter disease duration (28.9% vs 45.5% had OA for >5 years) and were taking more non-steroidal anti-inflammatory drugs (66.7% vs 49.2%) than patients whose data were analysed (data not shown).
Item reduction
In total, 524 patient files were analysed at this step. Concerning the 33-item provisional questionnaire, no missing values occurred for 487 cases (92.9%).
Fifteen items were omitted after the item reduction process, resulting in an 18-item questionnaire (score range 0–100, box 1). We omitted three items with a ceiling effect (items 31–33 of the provisional questionnaire, see table S1 in online supplement); no item had a floor effect. Two pairs of items were highly correlated (Spearman correlation coefficients 0.7–0.8) and two items were omitted (items 12 and 24 of the provisional questionnaire, see table S1 in online supplement).
Box 1 The Knee Osteoarthritis Expectation Questionnaire
These questions assess your expectations concerning your physicians in the management of your knee OA. We ask you to score the degree of importance you attach to each item.
I expect that my physician:
Education
1 Tells me about the main side effects of the treatments prescribed for my knee OA
2 Explains how treatments for my knee OA work
3 Explains how to limit the impact of my knee OA on my daily life
4 Informs me about all the treatment modalities including complementary medicine
5 Explains why he/she prescribes or not laboratory and x-ray examinations
6 Explains the relationship between ageing and OA
7 Explains the relationship between my job and my knee OA
8 Explains to me what sports or work activities I can do
9 Informs me about the possibility of the involvement of other joints
10 Communicates with all the therapists involved in managing my knee problem
11 Takes into consideration my treatment preferences when prescribing
Human and technical supports
12 Helps me to get assistive devices that may be helpful for my daily life
13 Explains how to get help for activities of daily living that I have a problem with
14 Explains to my relatives my knee problem
15 Helps me to obtain psychological support
Physician's empathy
16 Takes time to listen to me
17 Takes into consideration all the problems I have in my daily living activities because of my knee
18 Takes time for my knee examination
Response modalities
0: Totally unimportant
1: Unimportant
2: Rather unimportant
3: Somehow important
4: Very important
5: Extremely important
OA, osteoarthritis.
Exploratory factor analysis extracted three main factors with eigenvalues of 11.81, 2.81 and 1.56 explaining 46% of the variance (figure 3). Each factor was easily characterised, factor 1 (11 items) representing expectations for education, factor 2 (four items) expectations for information on human and technical support, and factor 3 (three items) expectations for physician empathy. Another 10 items (items 4–8, 17, 19, 22, 26 and 28 of the provisional questionnaire, see table S1 in online supplement) were eliminated because of weak correlation (<0.5) in each factor.
Exploratory factor analysis of Knee Osteoarthritis Expectation Questionnaire.
Validity of the questionnaire
The reliability of the questionnaire was excellent, with a Cronbach α coefficient of 0.91 (95% CI 0.89 to 0.92). The reliability of each factor was good, with Cronbach α coefficients of 0.90 (95% CI 0.84 to 0.98) for factor 1, 0.85 (95% CI 0.82 to 0.87) for factor 2 and 0.79 (95% CI 0.74 to 0.83) for factor 3. Confirmatory multi-trait analyses confirmed higher intra-factor than inter-factor correlations (figure 4).
Confirmatory factor analysis of the Knee Osteoarthritis Expectation Questionnaire: distribution of intra-factor and inter-factor correlation.
Expected divergent validity (low correlations) was observed with knee pain score (r=0.19), WOMAC function score (r=0.22) and physical and mental scores of the SF-12 (r=−0.07 and r=−0.22, respectively) (table 2).
Divergent validity of the Knee Osteoarthritis Expectation Questionnaire: correlation of global score and other outcome measures
Test–retest reliability
Test–retest reliability was good, with an ICC of 0.78 (95% CI 0.60 to 0.89). With the Bland–Altman technique, the mean difference was close to 0 (0.36 (1.29)). The variability was random and uniform throughout the range of values (figure 5). With the limits of agreement, 95% of the differences between the two measurements could be expected to lie between −14.4 and 15.1 points (2 SD of the mean difference). From the value of the SEM, the SDC for the global score was 14.3 points which, in a maximum of 90 points, equates to a 15.9% score change (see table S2 in online supplement).
Bland–Altman analysis of test–retest reliability.
Scoring of the KOEQ
According to factor and reliability analyses, an overall score can be used (0–90). The global score is obtained by adding the scores of each of the 18 items. The distribution of the global score and the distribution of the scores for each of the 18 items of the final and 33 items of the provisional scales are shown in figure S1 and table S3 in the online supplement. The metric properties of the global score should be tested in another sample of patients with knee OA.
Discussion
To our knowledge, this is the first questionnaire developed to assess patient expectations for the management of knee OA by their physicians. A questionnaire assessing expectations was developed several years ago for patients undergoing hip replacement,11 ,34 but no instrument exists to assess the expectations of patients with OA at an earlier stage.
The main strength of this study is the design used to generate the items. The in-depth interviews about patients' views of knee OA and its management provide a relevant qualitative database to select items that really matter to patients when building a patient-reported questionnaire. This approach is strongly recommended by the US FDA20 because it increases the content validity of an instrument. Most patient-reported outcomes widely used in OA were developed in the 1980s and 1990s, mainly by selecting items from expert viewpoints and/or pre-existing questionnaires.35,–,39 Because of discrepancies between patients and physicians in terms of what is important,1,–,3 the content validity of these questionnaires is questionable.
The Delphi design used to select items from the qualitative study is a classic recommended method22 that allows experts to give their opinions with blinding to other experts' opinions in a first step and to achieve consensus anonymously in a second step. The method prevents a ‘leading-expert’ effect. Items generated in the provisional questionnaire seemed understandable and acceptable, as seen by the low rate of missing answers for the provisional scale.
We translated the provisional rather than the final questionnaire into English to let researchers from other English-speaking countries test the item reduction step in their own country if they felt it could be relevant in a different background.
During the Delphi procedure the five-domain structure proposed by the steering group of the study to help experts select items was rapidly simplified into two domains—expectations about physicians and treatments—which suggests that French patients with knee OA structure their expectations this way. During the item reduction process all items about treatment expectations were omitted because of a ceiling effect. The fact that patients with knee OA have high expectations of treatment efficacy for pain, function and disease evolution is shared, logical and obvious and probably does not need to be recorded. The final questionnaire therefore focuses on patient expectations about physicians.
The metric performance of the questionnaire is promising. The instrument has excellent internal validity (reliability) and test–retest reproducibility. It probably has satisfactory construct validity because we observed the expected divergent validity and the factorial structure seems robust, with three factors identified and easily characterised after exploratory factor analysis and confirmation by confirmatory factor analysis. Patient expectations about physicians are organised around three axes: education, information about human and technical support and physician empathy.
This study has some limitations. The main limitation is that we did not include patients in the group of experts for the Delphi procedure. Although knee OA is a frequent clinical situation, no patient association exists in France and the identification and selection of patients implicated in the disease and its management is not easy. In addition, the questionnaire was developed in a strict French context and its content validity should be verified in other groups of patients with different cultural backgrounds. However, French society is highly multicultural, and this limitation applies to every patient-reported outcome because none of these outcomes has been developed simultaneously in different countries with different languages and cultures. For assessment of the validity of the questionnaire, we assessed divergent validity but not convergent validity. However, no other instrument exists to assess the expectations or a concept close to expectations in this context. Finally, our item selection and reduction processes resulted in a questionnaire assessing only patient expectations about physicians, which does not cover the whole range of patient expectations. However, an instrument focusing on expectations about physicians may help to optimise patient–physician relationships, which probably play a pivotal role in the management of chronic diseases.3 ,8 ,9 ,15 ,40
This questionnaire may be helpful in two different approaches: (1) a qualitative individualised analysis of responses in routine practice that may help to increase the quality of the patient–physician relationship by providing relevant information to the physician to adapt attitudes, educational messages and treatment strategies to the patient's expectations; and (2) a quantitative analysis that may provide useful information in clinical research on the effect of high or low levels of expectations or their changes or fulfilments of patient satisfaction, treatment compliance and outcomes and disease evolution.
In conclusion, we propose a new 18-item patient self-reported questionnaire assessing patient expectations for the management of knee OA by physicians. This questionnaire has robust metric properties, particularly content and construct validities. Its usefulness in the clinic and for clinical research remains to be demonstrated.
Acknowledgments
The authors thank Pierre Alainguillaume, Philippe Anract, David Biau, Xavier Chevalier, Dominique Desjeux, Francis Guillemin, Alain Lorenzo and Pascal Richette for their participation in the Delphi process.
References
Footnotes
-
Funding Pfizer France.
-
Competing interests None.
-
Patient consent Obtained.
-
Ethics approval The study protocol was approved by the ethics committee of Cochin Hospital, Paris. Investigations were conducted according to the principles of the Declaration of Helsinki.
-
Provenance and peer review Not commissioned; externally peer reviewed.