Several recent trials with the anti-tumour necrosis factor α (anti-TNFα) agents infliximab^1–5 and etanercept^6–8—both of open label type and double blind, randomised, controlled—have suggested that these agents are effective in active ankylosing spondylitis (AS). TNFα blocking agents are generally considered to be a major breakthrough in the overall treatment of this disease. In contrast to rheumatoid arthritis, Crohn’s disease, and psoriatic arthritis—in which corticosteroids and disease modifying antirheumatic drugs (DMARD) have established efficacy—such treatment has only limited value in AS.⁹ In fact, at present the treatment of AS consists mainly of physiotherapy, non-steroidal anti-inflammatory drugs (NSAIDs), local corticosteroid injections, and DMARD therapy with sulphasalazine in patients with peripheral arthritis.

When the first randomised trial with infliximab¹ was designed in 2000, there were no established improvement criteria in AS at all. From our pilot study we knew about the strong efficacy of anti-TNFα therapy with infliximab in active AS.^10,11 We therefore chose 50% improvement in the Bath ankylosing spondylitis disease activity index (BASDAI)¹² as the major outcome and improvement criterion, in analogy to the American College of Rheumatology (ACR) 50% criterion.^13,14 This was also used in the subsequent German randomised controlled trial (RCT)¹ and it has frequently been adopted in later trials by other investigators.^2–7 The consensus conference on anti-TNFα treatment in AS which took place in January 2003 in Berlin included 50% improvement of BASDAI in its recommendations for discontinuation of anti-TNFα treatment in clinical practice.¹⁵

The Assessments in Ankylosing Spondylitis (ASAS) working group published a preliminary definition of short term improvement in AS based on the best discrimination between NSAID treatment (50%) and placebo (25%).¹⁶ These ASAS criteria are based on improvement of at least 20% and 1 unit (on a 0–10 scale) in three of the following four domains: patient’s global assessment, pain, function, and morning stiffness. In addition, it is required that there is no worsening by more than 20% in the remaining domain. It was unclear whether the ASAS improvement criteria were also be the best criteria to assess the efficacy of anti-TNFα treatment. There are some reasons to consider why this might not be so. Initial NSAID trials are always based on the flare design: patients who are in need of NSAIDs are asked to stop the drug they are taking and only if they show a flare they can be randomised. The trial design in the anti-TNFα studies has been completely different. Patients with very active disease despite optimal doses of NSAIDs are included in the trial. Even in this group of patients with very severe disease, unresponsive to NSAIDs, there is a major treatment response of anti-TNFα therapy. The difference in trial design might also have implications for the placebo response and the regression to the mean effect.

Another issue is the choice of domain. In NSAID trials the domain spinal mobility, which seems rather important for patients, proved unresponsive to treatment, and the same was true in the acute phase reactant trials. Anti-TNFα therapy has a positive effect on both domains, which are part of the ASAS core set to assess treatment with a presumed DCART (disease controlling anti-rheumatic therapy) effect. Therefore it was necessary to repeat the process of developing improvement criteria for this specific aim and to compare the novel criteria with the ASAS 20 and the often used BASDAI 50.

The domains used for the improvement criteria are based on the core set of outcome measures recently proposed by the ASAS working group^17,18 including validated measures of function,¹⁹ spinal mobility,²⁰ and acute phase reactants. This study provides data that will enable us to choose the best set of improvement criteria to assess changes in AS.

METHODS

Choice of domains for outcome measurement in AS

Six different outcome domains were chosen. The first four are already included in the ASAS improvement criteria, which were derived from the ASAS core set: pain, patient’s global assessment, function, and morning stiffness. These domains were assessed by a visual analogue scale (VAS) for pain and patient global assessment, by the Bath ankylosing spondylitis functional index (BASFI)¹⁹ for function, and by the two last questions of the BASDAI for morning stiffness.¹² Two additional domains were added in this analysis: spinal mobility and acute phase reactants. These are also included in the core set for DCART.¹⁷ The tools that were used to assess these were the Bath ankylosing spondylitis metrology index (BASMI)²⁰ and C reactive protein. The BASMI comprises semiquantitative assessments of anterior lumbar flexion (modified Schober), lateral lumbar flexion, cervical rotation, occiput to wall distance, and intermalleolar distance.

On the basis of these six domains, many different candidate criteria for anti-TNFα treatment in AS were developed. The first set of criteria was based on the primary list of all six domains, with the following modifications:

• The domains pain and morning stiffness were replaced by the BASDAI,¹² the most frequently used measure for disease activity in AS, which includes both these domains but also assesses other important features of AS such as peripheral arthritis, enthesitis, and fatigue.

• The domain acute phase reactants was excluded because it is not entirely clear if acute phase reactants reflect disease activity reliably, and they are not increased in all AS patients.²¹

• The domain spinal mobility was assessed by the single item examination of lateral lumbar flexion instead of the whole BASMI. This was because lateral lumbar flexion alone had the highest standardised response mean (SRM) among all five individual BASMI items. Thus we assumed this combined measure performed as well as the total BASMI (see below) but was easier to do. Chest expansion was not assessed in the trials.

Ways of defining improvement

The different sets of domains were combined to form Boolean-type improvement criteria. Such criteria require improvement in all or at least a specified subset of the domains at a certain specified level. ACR response criteria for rheumatoid arthritis are an example of Boolean-type improvement criteria. They include criteria requiring improvement in all domains or in, for example, four of six domains. Finally, we defined additional candidate improvement criteria with mandatory improvement in one of the domains. For this, we selected three domains of high clinical relevance for the evaluation of efficacy of anti-TNFα treatment: pain, function, and spinal mobility (for example, a change in any five of six domains would include a mandatory improvement in spinal mobility). We show only selected results of all the candidate improvement criteria investigated.

Improvement in a single domain was determined by a combination of relative and absolute change from baseline values or by a relative change alone. Relative changes at levels of ⩾20%, ⩾30%, ⩾40%, ⩾50%, and ⩾60% improvement were chosen. If relative and absolute changes were combined, the following combinations were applied: ⩾20% change plus 1 unit on a 0–10 scale, ⩾30% change plus 1 unit, ⩾40% change plus 2 units, ⩾50% change plus 2 units, and ⩾60% change plus 3 units, respectively.

In some of the recently published trials on new treatments for AS, response was defined as improvement in the single domain disease activity as assessed by the BASDAI using cut off values between 20% and 70%. In order to compare these with the multiple domain improvement criteria defined above, we calculated the 20% to 70% improvements in the BASDAI response in steps of 10%. Additionally, we calculated the ASAS improvement criteria for several cut off values ranging from 30% to 70% for further comparison. For this, the original ASAS 20 improvement criteria were modified in the following ways: an ASAS 40%, 50%, 60%, and 70% response was defined as a relative improvement in three of four domains, with an absolute improvement of at least 2 units (3 units) and no deterioration in the remaining domain.

Patients

The first dataset to evaluate candidate improvement criteria contained data from week 12 of the placebo controlled part of the “infliximab in AS” trial with 69 active AS patients,¹ who were treated with either 5 mg/kg infliximab or placebo in weeks 0, 2, and 6. The discriminant properties of the candidate improvement criteria were validated with a second dataset which included results from the “etanercept in AS” trial.⁸ In this trial patients received 25 mg etanercept or placebo subcutaneously twice weekly. The clinical efficacy of infliximab and etanercept was rather similar. In the etanercept trial fewer patients were included (n = 30) and the placebo controlled phase was only six weeks long, as compared with 12 weeks in the infliximab trial. In both trials the patients were selected using the same inclusion and exclusion criteria (table 1). Only patients with high disease activity—defined by a BASDAI value of at least 4 and pain of at least 4 on a VAS despite treatment with NSAIDs—were included in both studies, in order to ensure that these patients were suitable candidates for treatment with the anti-TNFα agents. The patients in both trials were in a very active state of disease, with mean BASDAI values at baseline of around 6.5 in both trials.

Statistical evaluation

A selection was made to determine the best performing newly developed improvement criteria using the data from the infliximab trial, based on the following rules.

(1) Criteria with a placebo response above 10% were excluded. This cut off was chosen arbitrarily by expert opinion in order to omit improvement criteria with a high placebo response from further evaluation and to select for other improvement criteria with a high specificity to identify patients who respond to anti-TNFα therapy. This rule was determined by calculating 95% confidence intervals of the placebo response rates. Thus improvement criteria with a significantly higher placebo response than 10% could be identified.

(2) Criteria with a low placebo response (below 10%) and a high power to detect the difference between placebo and effective treatment were considered further. The power of the different criteria was evaluated by comparing their χ² test values. By bootstrapping analysis 95% confidence intervals of these χ² values were calculated. As there is no exact subset selection procedure available for dependent χ² values, an approximate bootstrap procedure was applied to investigate which criteria were significantly different from those with the highest χ². For that purpose all improvement criteria not already excluded because of the high placebo response were compared with the improvement definition that had the highest power (highest χ² value; reference criteria). All criteria with a significantly lower power than the reference criteria were excluded.

A validation step was carried out by comparing the 12 week results of the infliximab trial with a combined sample of six week data from the infliximab and etanercept trials. In four patients withdrawn from the trial before week 12 the “last observation carried forward” method was applied to estimate the 12 week data by their six week data.

RESULTS

Candidate improvement criteria were evaluated using the data from the infliximab trial.¹ Most of these were very strict—only four of 50 candidate improvement criteria shown in tables 2 to 4 had placebo response rates that were significantly higher than 10% (Nos 7, 39, 40, and 45). These criteria were removed from further consideration.

Of the remaining criteria, No 1 (table 2) performed best among those that were based on the original six domains (χ² = 31.9 (95% CI, 18 to 47)). This high discriminative power is based on a very low placebo response of 2.9% and a high response in the group treated with infliximab of 67.7%. It was used as the reference criterion to investigate which of the others differed significantly from improvement criterion No 1.

There were two criteria with slightly higher χ² values, but these did not include the original set of six domains and did not differ significantly from criterion No 1, so they were not considered as reference improvement criteria. In detail, these two improvement criteria combined five domains because the domain acute phase reactants was omitted. The first criterion (No 28, table 3) defined improvement by a ⩾20% change in four of five domains (χ² = 32.6) and the second (No 29) was a modification of the first by adding a mandatory improvement of pain (⩾20% change in four of five domains including pain; χ² = 32.6). Furthermore, candidate criteria were omitted from analysis when their χ² values were significantly lower than reference criterion 1 (table 2) by bootstrap analysis. This resulted in 36 improvement criteria with a high discriminative power to detect differences between placebo and treatment with infliximab, which are marked by an asterisk in tables 2, 3, and 4.

In more detail, the following conclusions can be drawn from the calculations done with these 36 improvement criteria using the data from the infliximab RCT.

Improvement criteria Nos 4, 5, and 8 (table 2)—which defined improvement in at least three or four of six domains—had a similar power for discriminating between the active drug and placebo as reference criterion No 1. However, the results of these improvement criteria mainly reflected the original ASAS domains (pain, patient global, function, inflammation) and not the more objective measures of C reactive protein and spinal mobility. For example, 75% of the responders of improvement criterion No 4 responded in all of the four original domains and only 42% in C reactive protein or spinal mobility. For improvement criterion No 5 the figures were 65% and 39%, respectively.

Improvement criteria which defined improvement only by a relative change—for example, criterion 1 (table 2)—performed as well as the corresponding improvement criteria where relative and absolute improvements were combined (as in criterion 10 (table 2)). This was mainly because the baseline values were high and consequently a (large) relative improvement immediately results in a corresponding (high) absolute value. Based on feasibility, improvement criteria with definitions dealing only with relative change are preferable to improvement criteria reflecting relative and absolute change.

Improvement criteria with combinations of different values of relative improvement—for example, criterion 19 (table 2) with ⩾40% in four of six domains or ⩾30% in five of six domains—and those with combinations of different values of relative improvement and absolute improvement showed very similar results to the corresponding more feasible improvement criteria (Nos 2 and 5). Moreover, they showed somewhat lower responses in the group treated with the active drug than were found with reference criterion 1, so these improvement criteria performed worse for defining response.

We also analysed another subset of candidate improvement criteria with mandatory improvement in a specific domain that was considered to be highly relevant clinically (pain, function, and spinal mobility). Improvement criteria with a mandatory improvement in pain (Nos 17 and 18, table 2) had a similarly good performance to those without this restriction (Nos 1 and 5, table 2). Thus a mandatory improvement in pain is not necessary for a definition of improvement with anti-TNFα drugs. Improvement criteria with a necessary improvement in domain function and spinal mobility showed a worse performance, with significant differences in their χ² values compared with reference criterion 1 (data not shown), indicating that a mandatory improvement in function or spinal mobility had less statistical power.

With differently modified improvement criteria—where disease activity was assessed using the BASDAI instead of the domains pain and morning stiffness, resulting in improvement criteria with five domains (table 3)—the performance was nearly as good as with six domains (table 2). In detail, criterion 23 in table 3, with a definition of ⩾20% change in any four of five domains including the BASDAI, had a slightly less good χ² value (28.6) than the reference improvement criterion of ⩾20% change in any five of six domains (χ² = 31.9). Because these differences might be from chance alone, it can be assumed that criteria with inclusion of the BASDAI had a similar good performance to criteria with the whole set of all six domains.

In candidate improvement criteria where acute phase reactants were omitted—resulting in criteria with five domains—similar results were obtained to the corresponding criteria where acute phase reactants were included (table 3). This shows that inclusion or exclusion of these variables had no influence on the performance of the improvement criteria. Candidate improvement criteria included the domain spinal mobility, as assessed by a single measure instead of the whole metrology index (the BASMI), which is a combination of five mobility measures. Standardised response means (SRM) were calculated using the data from the infliximab trial (comparing baseline with week 12) for all five measures of the BASMI. It should be mentioned that all five BASMI measures are semiquantitative; their ranges, in centimetres or degrees scored as 0, 1, or 2, were used for calculations of the SRM values. This indicates that a change of, for example, 1 cm or 10% from 10 to 11 for the lateral lumbar flexion corresponds to a change from 0 to 1 on the semiquantitative scale, which would be a 100% change. Linear scales for the five measures would be better but were not available in the trial datasets and therefore could not be used.

Lateral lumbar flexion, with an SRM of 0.84, showed the best representation of the total BASMI, followed by anterior lumbar flexion (Schober test) with an SRM of 0.64. All three other measures (cervical rotation, tragus to wall distance, and intermalleolar distance) worked less well, with low SRMs. We therefore tested improvement criteria in which spinal mobility was assessed with this single mobility measure alone. The data show that the χ² values were as good as those obtained with the whole BASMI (table 3).

In both trials, about 35–50% of patients had normal values at baseline ( = 0) and another 30–40% had a flexion of 5–10 cm ( = 1). This indicates that almost half the patients could not improve because of initially normal measurements.

When we tested improvement criteria already used in recent trials with anti-TNFα agents, the highest χ² value of 23.5 was found for a ⩾40% change in BASDAI (table 4). This value was somewhat lower than, but not significantly different from, reference improvement criterion 1 (table 3). With a cut off of 40%, the ASAS response criteria showed similar results, as indicated by a χ² value of 26.5 (table 4).

Validation of the candidate criteria using the second combined dataset from the etanercept trial showed that most of the best performing candidate criteria from the infliximab dataset also had the highest χ² values in the second combined dataset (tables 2 to 4), indicating good reliability of the set criteria.

DISCUSSION

As anti-TNF agents are internationally considered to represent major progress in the treatment of AS, and as they seem to be far more effective than NSAIDs, it is apparent that clinically relevant criteria are needed that perform better than those developed for assessing NSAIDs.¹⁶ The present analysis of data on almost 100 patients treated with the anti-TNFα agents infliximab and etanercept was likely to provide a sound basis for the development of improvement criteria for anti-TNFα therapy in AS. We therefore analysed the data collected in our randomised placebo controlled trials on infliximab¹ and etanercept⁸ so that we could propose improvement criteria for anti-TNFα agents in AS. As the sample size in this study was not extensive, we compared the candidate criteria not only by their χ² values but also by the 95% confidence limits of the χ² values, and applied a statistical subset selection procedure to analyse the differences between candidate criteria in relation to the reference criterion.

On the basis of the data from both trials with anti-TNFα agents in AS, we propose the following two improvement criteria for further consideration: a ⩾20% change in five of six domains, and the 40% modification of the ASAS response criteria. As neither of these two sets is clearly superior on statistical grounds, the final decision needs to be taken by expert opinion. The performance of these two proposed improvement criteria should be further tested in other datasets from ongoing trials with both these anti-TNFα agents.

The two sets of improvement criteria are different in several aspects: one uses a cut off of 20% improvement in five of six domains, including spinal mobility and C reactive protein as more objective measures, while the other takes advantage of the already established ASAS improvement criteria but increases the cut off to 40% improvement in three of the four domains: patient’s global, pain, function, and morning stiffness. In terms of simplicity, the ASAS 40 improvement criteria may have the advantage.

In contrast to treatment with NSAIDs, anti-TNFα therapy is very likely to have disease controlling properties, so assessment of improvement in this area of treatment might well include domains for assessment of function and inflammation such as spinal mobility and acute phase reactants. These domains are known not to be influenced by NSAIDs to a large extent,¹⁶ while both are known to be influenced by anti-TNFα treatment.¹ Thus including these measures seems advantageous when mean group levels are compared. However, when looked at in more detail and in individual patients, the problem arises as to whether a 20% improvement in C reactive protein or spinal mobility is a reliable cut off point. For example, can we be sure that a patient whose C reactive protein has improved from 10 mg/dl to 8 mg/dl (a 20% improvement) is really obtaining benefit? And what about the patient with a 20% increase in lateral spinal flexion (for example from 10 cm to 12 cm)? From our RCT data we cannot answer these questions because all measurements were part of the BASMI, which means that they were less precise owing to the semiquantitative system involved. This implies that every improvement is by definition a 50% or a 100% step. Furthermore, as shown in the results section, C reactive protein could be taken out of the improvement criteria set without loss of information. Clearly, C reactive protein will not be helpful in patients with active disease but with low or normal C reactive protein levels. These examples indicate that 20% improvement, even of more “objective” indices, may not be the ultimate solution for an optimal set of improvement criteria. At the other hand, these are the only objective domains that have high face validity as being important in controlling the disease process. Although 20% appears to be a small improvement, it is difficult to achieve a consistent improvement across at least five different domains. Moreover, the available data are from six to 12 week trials. So if an improvement of spinal mobility of this magnitude can be achieved in this short treatment period it is rather impressive and has not been matched by any other treatment. Clearly, more data on the impact on long term outcome need to be established.

The ASAS improvement criteria cover only improvements in signs and symptoms of the disease, because other measures (including spinal mobility) are not sensitive to change in AS patients treated with NSAIDs. All four domains of the ASAS improvement criteria are assessed by patient questionnaires which are subjective by their nature. The original ASAS 20% improvement criteria, as developed in and for trials with NSAIDs, had a rather low sensitivity (50%) but a higher specificity (75%) for showing improvement in AS patients.¹⁶ These values indicate that only 50% of the responders to NSAIDs are detected by these criteria but also that 25% of the non-responders to NSAIDs are falsely identified as responders. In a subsequent paper, patients and physicians considered that the response rate is underestimated by using these criteria, but that patients who are judged to be responders according to the criteria are indeed responders.²² However, among a whole range of possible criteria these ones clearly performed best.^16,22

As shown in the results section, the situation of the ASAS 40% improvement criteria when used in anti-TNFα trials is clearly different, as more than 60% of the responders are identified and only 5% of responders were found in patients who had received placebo treatment. However, it should be emphasised that response rates obtained in NSAID trials cannot be compared directly with those obtained in anti-TNFα trials. NSAID trials use a so called “flare design”: patients already use NSAIDs before inclusion in the trial; they have to stop treatment; and only if there is a certain increase in symptoms (for example, 30%) and a certain level of symptoms (for example, >4) will they be included in the trial. Thus the patients are already known NSAIDs responders. In contrast, in anti-TNFα trials patients have high disease activity despite the use of NSAIDs,

Nonetheless, considering the differences between NSAID and anti-TNFα trials, anti-TNFα therapy has a much greater efficacy than NSAIDs. This has been consistent in all trials published so far^1–7 in which high percentages of patients showed an improvement in disease activity of more than 50% in comparison to baseline. Accordingly, in most clinical studies on the efficacy of anti-TNFα therapy in AS, a 50% improvement in the BASDAI has been used as one of the main outcome variables.¹ The ASAS experts have recently recommended that the efficacy of anti-TNFα therapy should be monitored by measuring BASDAI in clinical practice and, as already mentioned, that consideration should be given to discontinuing it in patients whose response is less than 50%. Although the BASDAI 50% improvement criterion performed less well in this dataset than the two improvement criteria discussed above, it seems possible for it to be used in clinical practice because it is well known and easy to perform.

One important additional aspect of this study was the finding that the lateral spinal flexion test alone seems to represent the total BASMI which comprises four other measures. This suggests that it might be sufficient to use this tool to assess spinal mobility in AS trials with anti-TNFα agents. However, it needs to be stressed that the BASMI does not include continuous measures but subscales (for example >20 cm = 0, 10–20 cm = 1, <10 cm = 2). Furthermore, chest expansion was not included in these analyses, a variable that has also been found to be sensitive to change during anti-TNFα trials.⁵

Conclusions

In all, this analysis showed that two sets of improvement criteria are useful for assessing improvement in AS trials. No final decision about which set to use was taken after an expert session of the ASAS working group in October 2002. There will be further possibilities to study the performance of these criteria in the large clinical trials of anti-TNF treatment which are now ongoing. Lateral spinal flexion seems to be a useful instrument for measuring spinal mobility in AS patients.