Statistics from Altmetric.com
Greenwood et al have questioned the usefulness of the Stanford Health Assessment Questionnaire (HAQ) in the clinic.1 Using Bland and Altman's limits of agreement procedure,2 they report that an HAQ score change of at least 0.48 units is required to confidently reflect significant change. We have used the HAQ in the care of our patients for many years and think that their conclusions about the extent of change required and the non-usefulness of the HAQ in clinical care is not correct.
We have replicated the authors' study in paired visits of 443 patients with rheumatoid arthritis (RA) followed up in the clinic by of one of us (FW), and in a survey databank of 2720 patients. Our data are in general agreement with the authors' data.
Using the limits of agreement procedure, we found that the 95% confidence intervals (CI) for the HAQ were approximately 0.9 for clinic data and 0.6 for survey data. But we also found the following. To exceed 95% CI, changes in excess of 40 mm/1st h for the erythrocyte sedimentation rate (ESR), 70 mm Hg for grip strength, 10 joints for a joint count, and four units on 0–10 VAS scales for pain, global severity, and fatigue were required.
Although clinicians may have trouble in interpreting HAQ scores, there is no doubt that one does not need a change of 40 mm/1st h for ESR, 70 mm Hg for grip, or 10 joints to detect clinical change. These data, which indicate that we cannot reliably do what we are already reliably doing, suggest two interpretations. Firstly, stable measures of overall health are themselves noisy and do not reflect changes in arthritis clinical status. Secondly, they suggest that the Bland-Altman method for agreement and reliability may not work well and may not have a simple interpretation when extrapolated to settings such as these.
In additional analyses that measured relative accuracy and precision, we examined all of the above variables as well as the SF-36 and WOMAC variables using Lin's concordance coefficient.3 The HAQ was the most accurate and precise of all of the above variables, with concordance correlations of 0.809 and 0.902 in the two analytic sets.
We also believe it is a mistake to use the HAQ in isolation. One would not treat RA with a sedimentation rate or C reactive protein (CRP) or joint count alone, and the HAQ should not be used in that manner. In our clinics we collect HAQ, pain, global severity, joint counts, and ESR/CRP, and we use all of these data together to understand the status of the patient. One simple way to do this is to is to assign –1 or 0 or +1 to respective negative, neutral, and positive changes in these variables, and then to add up the individual values. Scores of three or above are associated with significant, clinically important change. Clinicians generally go further and look at the magnitude of the changes to improve further the usefulness of these tests, and for research purposes it is possible to develop a combined standardised score that documents these changes. But clinicians are also Bayesians, and use the HAQ and other clinical measures with knowledge of the particular patient and his longitudinal pattern of responses for each separate variable. We suggest that when the HAQ is used in this manner it provides continuing clinically useful information.
Scientific research in chronic rheumatic diseases will benefit when the clinical investigator brings to the research table the same longitudinal, Bayesian, non-linear insights that the clinician has long had and has long used. The HAQ works in the clinic when used and interpreted appropriately.
We are pleased that Drs Wolfe, Pincus, and Fries have replicated our findings about the level of variability in the Health Assessment Questionnaire (HAQ) for individual patients. We have found the HAQ to be a very useful measure of group outcome. However, in the clinic we work with individual patients and the purpose of our study was to quantify the magnitude of HAQ score change which would be needed reliably to reflect significant change from a patient's perspective. The calculated confidence intervals formally quantify the observation that an HAQ score can change by a considerable amount over just a short period of time, in the absence of any change in health perceived by the patient. The method evaluates expected levels of agreement under defined circumstances (in our case no change in health perceived by the patient over two months). Different levels of confidence can be chosen according to requirements. However, the sensible application of the method depends on giving serious consideration to the nature of the concept being measured and how the results are paired.
We too have found a similar range of variability for individual patients, in a whole range of standard outcome variables, which we would suggest is a function of their subjectivity and consequent susceptibility to other factors, such as mood. However, conventional outcome measurement remains a useful tool for recording group progress over time.
We quite agree that in clinical practice it would be unreasonable to attempt to use any measure in isolation. However, in order to integrate multiple pieces of evidence in a Bayesian manner it is necessary to have some idea of the probabilities involved. We set out to provide such information for the HAQ with a view to assisting clinicians in their interpretation of HAQ score changes in the light of all other available evidence. However, in the event, the amount of subjective variability was such that we feel it cannot reliably be used to influence or fully explain management changes for individual patients.
In an era when clinicians are having to argue for access to new and expensive treatments such as infliximab and etanercept we need an outcome measure which will reliably reflect individual responses over relatively short time intervals (perhaps 6–12 months), and practising clinicians need to understand the limitations of the measurement tools they use.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.