PATIENTS AND METHODS

Design

The study was designed as a controlled clinical trial with patients serving as their own controls. A three month control period, where disease activity was assessed without feedback, was followed by a 12 month period, where feedback to the rheumatologist was provided. It was suggested that the control period would provide an estimate for a stable course of disease activity, and that disease activity would show a reduction in the feedback period as compared with the control period. As a consequence of reduced disease activity, it was expected that the development of disability and joint damage during the feedback period would remain stable, or even improve.

Recruitment

Rheumatologists from the rheumatology departments of the five university hospitals, six regional hospitals, and from two rheumatological practices throughout Switzerland agreed to participate. Patients were recruited during 1997–98. The rheumatologists asked consecutive patients with RA (according to the American College of Rheumatology criteria) to participate in self assessment of disease activity for a period of three months.

Control period

When the patients agreed to participate, they were sent the rheumatoid arthritis disease activity index (RADAI) questionnaire on signs and symptoms in RA, once a month, for three months.⁴ However, when according to the rheumatologist a change in DMARD treatment appeared to be necessary, the control period was stopped and the procedure for the feedback period started immediately.

Feedback period

The start of the feedback period was scheduled at the fourth month. The patient was informed by the rheumatologist about participation in the measurement feedback system. To be included in the study the patient had to provide written informed consent. In that case, the RADAIs from the control period were sent to the coordination centre and stored for later analysis. At the start the rheumatologists collected the following data: joint counts, erythrocyte sedimentation rate (ESR), current drug use, radiographs of the hands and feet (not older than six months), and patient assessed disease activity (RADAI), disability (Stanford Health Assessment Questionnaire (HAQ)),⁵ sociodemographic variables, and comorbidities. These data were then sent to the coordination centre, where the data were processed and a feedback report was returned within 10 days (fig 1). For the feedback period, the rheumatologists were advised to monitor disease activity either with every DMARD change or, at the least, every three months. For monitoring, disease activity was assessed by the rheumatologist (disease activity score (DAS28)),⁶ and by the patient (RADAI). An updated feedback report was sent automatically when the rheumatologist sent those data to the coordination centre. After 12 months, the last study visit was scheduled, which was identical to the starting visit.

Drop outs

Patients were excluded from the analysis if (a) a change in DMARD treatment in the control period was necessary (to avoid influencing the doctors the patients were excluded after completion of the study); (b) the assessments of the control period were missing.

Measurements

The rheumatologists were provided with standardised information on how to perform the joint counts and how to handle the questionnaires. At the coordination centre, the data were checked for completeness and appropriateness before entry. Ambiguities were solved by a telephone call.

The DAS28 was calculated from the results of the 28 swollen joint count, the 28 tender joint count, and ESR.⁶ The DAS28 ranges virtually from 0 to 10. A DAS28 <3.2 is regarded as low level disease activity, a DAS28 of 3.2–5.1 as moderate, and a DAS28 >5.1 as high level disease activity.⁷ The RADAI is a five item patient assessed questionnaire, including arthritis pain, past and current global disease activity, duration of morning stiffness, and a tender joint list.⁴ The RADAI ranges from 0 to 10, where higher values are indicative of higher levels of RA disease activity. The RADAI has been shown to be reliable, valid, and responsive for the assessment of disease activity in RA.^8–10 The pain item is an 11 item numerical rating scale. The disability index of the HAQ contains 20 questions about difficulties experienced with eight activities of daily living, and four questions about the assistance used to perform these.⁵ The HAQ is scored from 0 to 3, where higher values are indicative of more difficulties when performing activities of daily living. Joint damage was scored from radiographs of the hands and feet by readers unaware of the study allocation, using the Ratingen x ray score.¹¹ The scoring of the wrist joint was modified by scoring it as a single joint, instead of as four joints. The x ray score ranges from 0 to 160, where higher scores are indicative of more and larger erosions of the joint surface.

The patient provided sociodemographic information and information about comorbidities on standardised questionnaires.¹² All questionnaires were provided in the language preferred by the patient: German, French, or Italian.

Statistical analysis

Data from intermediate monitoring visits during the feedback period were not the subject of analysis, as need-driven visits can overestimate the levels of disease activity during this period. Consequently, there were five study time points: three in the control period, two in the feedback period.

The time course of the RADAI scores in the control period was compared with the course in the feedback period, using a continuous by class regression model with random coefficients (intercept and time effect) for patients.^13,14 The procedure thus accounts for repeated measurements on the same subjects. To account for the clustering of patients of individual rheumatologists, random coefficients (intercept and time effect) for rheumatologists were added.

Changes during the feedback period of disease activity (DAS28), patient perceived pain, disability (HAQ), and joint damage (x ray score) were analysed using paired t tests and 95% confidence intervals.

It was suggested in advance, that the results might be influenced by the level of disease activity in the control period, and whether measurement feedback was used during the feedback period. Accordingly, four subgroups were formed: (a) patients with low disease activity in the control period (RADAI score below the median) and no use of feedback (the rheumatologist had acquired no feedback reports); (b) low disease activity and feedback use (one or more feedback report acquired); (c) high disease activity in the control period (RADAI score of median or higher) and no use of feedback; (d) high disease activity and feedback use. The regression analysis and the analysis of before-after differences were repeated as subgroup analysis; differences between subgroups (contrasts) were tested using Scheffe’s procedure.¹⁵

To indicate DMARD changes during the feedback period, the drug at the start of the feedback period was compared with the drug at the end. The information from intermediate visits was not used, to prevent information bias through underreporting in the non-use group.

Data were stored in an Access 7.0 relational database (Microsoft Corporation, Redmond, USA) and analysed using SAS 8.1 (SAS Institute Inc, Cary, NC, USA). The research protocol was approved by the responsible Swiss medical ethical committee (UREK).

RESULTS

Sample

Forty eight rheumatologists enrolled 264 patients; 36 patients were drop outs (table 1). The RADAI scores and number of drop outs were similar between patients included by private practices, regional hospitals, or university hospitals. Reasons for dropping out were: because of DMARD change in the control period (n=33), and because all RADAIs in the control period were missing (n=3). No relevant differences on prognostic and outcome variables between drop outs and patients who completed the study were found (table 1). At the end of the feedback period, 38 patients were lost to follow up. These patients were not regarded as drop outs.

Response and feedback use

The response in the control period varied between 55% and 65% for each time point. At the end of the feedback period, 190/228 (83%) patients took part in the last study visit. During the feedback period, feedback reports were acquired by the rheumatologists for 142/228 (62%) patients. Of those, 90/142 (63%) patients had one feedback report, 52/142 (37%) had 2–5 feedback reports. For the remaining 86/228 (38%) patients, the rheumatologists obtained only the report that marked the start of the feedback period.

Was disease activity stable in the control period?

The group mean (SD) RADAI scores at the three time points in the control period were 3.6 (2.0), 3.7 (2.1), and 3.7 (2.1), respectively. According to the regression model, that is correcting for within-person dependencies, the RADAI scores did not significantly change over time: βtime = −0.008 (95%CI −0.12 to 0.11); p=0.89 (table 2).

Was disease activity reduced in the feedback period?

The random time effect for rheumatologists was omitted from the continuous by class model, because its associated variance did not significantly differ from 0 (not shown). The time effect in the feedback period was −0.067 per 30 days (table 2), which corresponds to a mean reduction of about 0.8 RADAI points over 12 months. The reduction in RADAI in the feedback period was statistically significant. However, the time effect in the feedback period was not significantly different from the time effect in the control period.

Subgroup analysis

The regression model was subjected to subgroup analysis; the results are shown in the lower part of table 2. The “level of disease activity” in the control period and “feedback use” in the feedback period were not associated, (χ²; p=0.81). The difference in time effects was significant in the subgroup with relatively high disease activity in the control period and feedback use in the feedback period (p=0.02). The time effect of that subgroup corresponds with a mean reduction in RADAI score over 12 months of more than three points. Between both subgroups with high disease activity (n=111), the contrast between the difference in time effects of the “use” and “no use” group was at the limit of significance: p=0.051.

How did disability and joint damage develop?

During the feedback period, disease activity (DAS28) and pain decreased, disability (HAQ) did not significantly change, but joint damage (x ray score) increased (table 3). The subgroup with high disease activity and feedback use, showed significant and favourable changes in DAS28, pain, and HAQ (table 3). Their increase in the x ray score was comparably small; it was largest in the subgroup with high disease activity and no feedback use. None of the differences (contrasts) between the subgroups were statistically significant.

Did changes in drug treatment occur?

The drug changes during the feedback period could only be registered from the patients who were not lost to follow up and had complete drug information (table 4). At the start of the feedback period, 32/228 (14%) patients had no DMARD treatment, and 20 (9%) patients had a combination treatment of two or three DMARDs. More than half of the prescriptions (59%) were unchanged over the feedback period. Change in DMARD treatment was not significantly associated with the level of disease activity (χ², p=0.33), or use of the feedback system (χ², p=0.50).

DISCUSSION

According to the results of this study, the use of measurement feedback was associated with a reduction of RA disease activity in the feedback period as compared with the control period, in patients with high disease activity. The RADAI reduction of that subgroup corresponds with a clinically important difference¹⁰ and was nearly three times greater than for the subgroup of patients also with high disease activity, but no feedback use. In concordance, the subgroup with high disease activity and feedback use showed an improvement in the HAQ score and only a small increase in joint damage. In contrast, the subgroup of patients with high disease activity and no feedback use, had no improvement in the HAQ score, and a progression in joint damage that was twice as large. Thus it appears that the measurement feedback system contributed to a reduction of disease activity in patients with RA.

However, it is clear that a measurement feedback system is not an intervention that causes health effects, but drugs may do so. If the assumption of a measurement feedback system for RA is that the system is used to evaluate whether disease activity needs to be better controlled, or to ensure that disease activity is still under control, then it does not seem to be adequate that changes in DMARD treatment took place in less than half of the patients. Moreover, changes in DMARD treatment were not related to the level of disease activity at baseline or to feedback use. As the study was not designed to include drug strategy in the analysis, and as it is very difficult to judge the appropriateness of specific DMARD management for individual patients, it cannot be concluded that the feedback did not influence decision-making. But the most important conclusion that can be drawn is that not many changes in DMARD treatment occurred, even in patients with high disease activity. Also, systematic monitoring with the feedback system was much lower than was originally expected. One of the reasons for the suboptimal use of the measurement feedback system may be that available treatment guidelines^16,17 were not explicitly incorporated. Possibly, not all rheumatologists regarded suppression of disease activity as an explicit treatment goal. Also, prescription habits might be influenced by the “pyramid paradigm”, which has now been discarded.^2,17

Another reason for the low use of the measurement feedback in the study may be the local healthcare system, where rheumatologists merely may have a consulting role for the general practitioner and see some of their patients with RA probably once yearly. Further, not all rheumatologists are used to outcome measures in clinical practice. Outcome measures are often appraised as “soft” data, unfamiliarity and difficulties with interpretation may lead to uncertainty if, and how, to use all the information.^18–21 But the measurement of RA disease activity and disease consequences has improved substantially, and is within the ability of practising rheumatologists.²² The reasons for use and non-use of the measurement feedback system are currently being studied.

The major advantage of the study design with a control period followed by a measurement feedback (“intervention”) period is that the patients and rheumatologists are their own controls, and thus are optimally comparable. An important limitation is that it was not possible to “blind” the participants. However, an attempt was made to keep the patients unaware of outcome expectancy, and therefore the primary outcome was assessed by the patient. A bias from “knowing to be observed” (Hawthorne effect) might have occurred in both the control and feedback periods, and thus would not have introduced differential bias. Owing to the fact that the rheumatologists were not “blinded”, prescription behaviour can theoretically be biased towards socially desired changes in DMARD treatment. However, as the number of patients with changes in DMARD treatment was low, this does not seem to have played an important part.

It must be noticed that the study of such a complex intervention as measurement feedback is difficult, mainly because (a) the intervention is mainly by the doctor, but important effects are expected in the patient; (b) the intervention is indirect, in the sense that in itself it will not influence disease activity; (c) disease activity is subject to many influencing factors at the same time—for example, on the levels of patient, treatment, treatment tolerance, social and physical surroundings, and prognostic factors of the disease; (d) the effects on outcome are expected to be relatively small, and the outcome measures used in RA may not be sensitive enough to detect small but relevant changes.

In primary care, several randomised controlled trials (RCTs) were performed on the effectiveness of computerised measurement feedback systems, with or without guidelines, mainly in patients having hypertension, diabetes, or anticoagulation need.^23–29 From these studies it appears that computerised decision support systems (CDSS) were generally ineffective in changing doctor performance or health outcomes, probably for the same reasons as mentioned above. In arthritis, we could identify two RCTs on measurement feedback systems, dealing with disability.^30,31 Neither trial found any health gain. The studies used feedback at fixed times, and thus the feedback did not systematically coincide with visits or actual patient needs. It is clearly an advantage if relevant information is available at the moment of decision, and decision options are clear. This is the case with measurement feedback on drug dosing, of which “titration” of disease activity in RA is an example. A systematic review of computerised drug dosing systems included five RCTs dealing with outpatient maintenance anticoagulation treatment,³² of which one trial provided evidence that quality of initiation and control of warfarin treatment was improved by CDSS in comparison with usual care.³³ From this it can be concluded that until now there is no strong evidence of the effectiveness of measurement feedback systems, computerised drug dosing systems, or computerised guideline implementation systems in several chronic conditions in primary care. However, there is still a strong argument for the adoption of a measurement feedback system, or some other form of CDSS, in RA. As the primary target of RA treatment is the control of disease activity,¹⁷ the treatment has to be individually adjusted depending on the treatment effect and limiting toxicity. For reasons of patient care alone, but especially when using expensive treatments, it is most appropriate to monitor and document drug use, treatment effects, and toxicity in the individual patient.¹⁷

The current study provided limited evidence that a measurement feedback system is effective in RA, but the system was not intensively used. The measurement feedback system should be optimised to facilitate its use in clinical practice and its effectiveness should be studied using a carefully designed RCT. For these objectives it should be kept in mind that the primary target of measurement feedback is the rheumatologist, not the patient. A much more intensive training on the use of a measurement feedback system, the DAS as target measure, and the inclusion of explicit treatment guidelines will be necessary to increase the clinical use of measurement feedback and, possibly, to reduce disease activity for a larger number of patients.