Medications that inhibit the activity of tumour necrosis factor alpha (anti-TNF) are the most widely used biological disease-modifying antirheumatic drugs for the treatment of severe rheumatoid arthritis (RA). Much has been written on the concern that the immune-modulating effects of anti-TNF may increase the risk of malignancy, infections and other serious adverse events.1^–6 Due to the relatively low incidence of these events randomised controlled trials (RCT) may be underpowered to detect specific risks. Larger populations for analysis are available from registries and post-marketing surveillance; however, registries are subject to many biases due to their non-random allocation of treatments and post-marketing data are not ideal due to underreporting, poor exposure estimates and the lack of a control group. Given the problems with these single sources of information, data pooling techniques play an important role in determining the relative safety of treatments.

Meta-analysis is powerful in detecting differences with relatively common events; however, several problems are encountered with rare events and when zero events exist in individual RCT. In these situations, meta-analysis is biased and confidence intervals can be so large that results can not be interpreted.7 Simple pooling calculates adverse event rates by adding the number of events observed in a specific exposure group across several RCT and dividing by the total exposure or number of subjects.

Pooling reduces the likelihood of zero event analyses. Pooling does not consider between-study variability and it may be susceptible to Simpson’s paradox (a statistical contradiction in which favourable results in several studies seem to be unfavourable when the studies are combined, or vice versa). This seemingly impossible result occurs when the studies used in the pooled analysis have imbalanced results or are combined using biased weighting variables.8

Given the problems with both the meta-analytic and pooled methods it is prudent to compare the results from each method when dealing with rare events. Our objective was to quantify the risk of serious adverse events important in the treatment of RA with anti-TNF therapies at recommended doses using meta-analytic and simple pooling methods. In addition, we attempted to provide practising rheumatologists with quantitative guidance on counselling their patients regarding the risk of serious adverse events with the use of biological agents to treat RA.

METHODS

Study selection

To be included trials must have been conducted in more than 30 RA subjects randomly assigned to either an anti-TNF or a control group (non-biological disease-modifying antirheumatic drug (DMARD) or placebo) over a minimum of 10 weeks. A Jadad9 score of two or greater was required for inclusion. Treatment arms of combination biological therapies were excluded.

Search strategy

MEDLINE, EMBASE and Cochrane databases were searched to 31 December 2007 using the terms adalimumab, etanercept, or infliximab and RA limited to English language, human clinical trials. Two authors (TRE and JPL) selected manuscripts to be retrieved for more detailed assessment. Disagreements were resolved by consensus.

Data abstraction

Anti-TNF groups were divided into drug and dose categories. Dose was defined according to the recommended maintenance dose from the product labelling (table 1). Only recommended and high doses were considered in the analysis.

The number of subjects experiencing death or at least one serious adverse event or serious infection was extracted for each treatment group. Extracting malignancy data from published clinical trial manuscripts requires caution as there is considerable variation in reporting, especially in the reporting of carcinoma in situ and non melanoma skin cancers.13 As manuscripts may aggregate malignancies differently, malignancies were allocated to three classes allowing for comparisons of similar outcomes: lymphomas; non-melanoma skin cancers and the composite endpoint of non-cutaneous cancers and melanomas. If a subject presented with two types of cancer, the cancers were allocated as a single event in the following order of priority: lymphoma > non-cutaneous cancer/melanoma > non-melanoma skin cancer. When the number of events instead of the number of subjects experiencing an event was reported, an assumption of one event per subject was made.

All data were abstracted as reported in the publications. If an event described in a publication could not be allocated to a particular time or treatment group other sources of information were used. All data were compiled by two authours (TRE and JPL) and disagreements were resolved by consensus.

Statistical analysis

Two types of meta-analyses were performed for events occurring during the controlled portions of the trials. The first calculated an odds ratio (OR) based on the number of subjects experiencing an event and the number of subjects receiving treatment in each group. The second method calculated a rate ratio adjusted for unequal follow-up times. When exposure was not reported it was estimated by assuming a linear dropout rate between time points at which subject disposition was provided.

A fixed effects model was chosen over a random effects model to quantify the overall effect of treatment because it often produces narrower confidence intervals with rare event data14 and we wanted to maximise our chance of finding an increased risk of adverse events. Consistency of treatment effects across the treatment groups was examined using the Q test; a p value of 0.10 rather than 0.05 was used to determine the presence of heterogeneity. In addition, the I² statistic was calculated with values over 50% considered heterogeneous. A random effects model was used if heterogeneity was found.

The Mantel–Haenszel method15 with Robins variance estimation16 was chosen over the Peto method as the base meta-analytic method because we anticipated imbalanced group sizes and the Peto approach biases towards the null in these situations.17 The inverse variance, DerSimonian, and risk difference methods are not recommended with rare event data.17 The reciprocal of the opposite treatment arm size was used as the base case continuity correction factor.18

For meta-analyses with rare event data it is recommended that several sensitivity analyses be performed.18 In addition to the base case, we used different methods to estimate risk (Peto method and a random effects model) and different continuity corrections (adding 0.5 to each cell and excluding trials with zero events) to total five or six sensitivity analyses per risk estimate.

A simple exposure-adjusted pooled analysis was performed using both controlled and uncontrolled portions of the trials. The total number of events reported was divided by the total exposure for a particular treatment group from all trials, a method commonly used in US Food and Drug Administration (FDA) reports and product labels.10^–12 18^–21 Events were allocated to the treatment the subject was receiving at the time the event occurred. The rate ratio was calculated as the event rate in the active group divided by the event rate in the control group. Confidence intervals and p values were calculated using a score-based method.22 In cases in which zero events occurred across all treatment arms of a specific group, the event rate was corrected for continuity by using the reciprocal of the exposure in the opposite treatment arm.

The time dependency of risk estimates was evaluated by performing a meta-regression on the log OR of recommended and high doses from the unadjusted meta-analyses against the duration of the study. The inverse of the Robins variance estimate was used for individual trial weighting and the maximum likelihood method was used to estimate between-study variance.

The primary analysis for each outcome was the effect of the recommended dose of each anti-TNF agent alone and anti-TNF together assessed by the meta-analytic and the simple pooled methods. The secondary analysis was to investigate the effect of high doses of anti-TNF agents. MIX 1.61 statistical software was used for the meta-analyses.46 SAS version 9.1 was used for all other calculations.

RESULTS

Search results

Twenty-three of 508 publications reporting the results of 18 individual trials, with 8808 subjects over 7846 years of follow-up included in the analysis (fig 1). Inclusion criteria, patient characteristics, control treatment and control duration differed across the trials (table 2). Overall, anti-TNF subjects had significantly longer follow-up times than controls: 307 versus 285 days (p<0.001) (table 3). In particular, the individual trials by Klareskog et al,23 Lipsky et al,24 Maini et al,25 Moreland et al,26 van de Putte et al,27 Weinblatt et al 28 and Weisman et al 29 had longer follow-up times in the anti-TNF groups (p = 0.009, p<0.001, p<0.001, p = 0.014 and p<0.001).

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Study selection

Deaths: recommended doses

One trial did not report on mortality;26 in the remaining studies 23 deaths occurred in 4097 subjects (0.6%) initially randomly assigned to recommended doses over 3800 subject-years. Two deaths occurred in 414 subjects originally randomly assigned to placebo who crossed over to recommended dose therapy and 11 deaths occurred in 2671 control subjects (0.4%) over 2124 subject-years (table 4). There was no evidence of increased mortality associated with any anti-TNF at recommended doses using the three risk estimation methods (table 5).

Serious adverse events: recommended doses

From the trials with published information on serious adverse events, 499 of 3581 subjects (13.9%) initially randomly assigned to recommended doses over 3032 subject-years experienced a serious adverse event compared with 257 of 2178 subjects (11.8%) over 1452 subject-years in the control group. No evidence of an increased incidence of serious adverse events was observed for any biological treatment at recommended doses using the three risk estimation methods (table 5).

Serious infections: recommended doses

Not all published trials reported the occurrence of serious infections. Of the trials with published information, 133 of 3729 subjects (3.6%) randomly assigned to recommended doses over 3714 subject-years and 72 of 2618 control subjects (2.8%) over 2116 subject-years experienced a serious infection. There was no evidence of an increased risk of serious infection associated with any anti-TNF at recommended doses using the three methods of risk estimation.

Malignancies: recommended doses

Not all publications reported the incidence of cancer. In 4099 subjects initially randomly assigned to recommended doses over 3805 subject-years, 34 subjects (0.8%) developed malignancies (12 malignancies occurred after crossover and 10 were unable to be allocated to the controlled or uncontrolled portion of the trial). Fifteen of 2672 control subjects (0.6%) followed for 2124 subject-years developed a malignancy. Overall, no increased risk was observed for lymphomas, non-melanoma skin cancers or the composite endpoint of non-cutaneous cancer and melanoma at recommended doses (table 5). Overall, the confidence intervals for the risk estimates were quite wide.

High-dose biological therapy

The clinical trials included in the analysis studied the effects of higher than recommended doses of adalimumab and infliximab, with 15% and 54% of subjects receiving high doses, respectively. No clinical trials evaluated high doses of etanercept. The mean dose of adalimumab in the high-dose groups was 49 mg/week, which is approximately 245% above the recommended dose, whereas the mean dose of infliximab was 1.16 mg/kg per week, or approximately three times the recommended dose. The unadjusted meta-analysis and the pooled method identified an increased risk of serious infection with high-dose anti-TNF therapy, OR 2.07; 95% CI 1.31 to 3.26 and risk ratio (RR) 1.83; 95% CI 1.18 to 2.85, respectively. Adjusting the meta-analysis for exposure produced evidence of heterogeneity and required the use of a random effects analysis, which did not produce significant results, RR 1.99; 95% CI 0.90 to 4.37. The risk of death, serious adverse events and malignancies was not increased with high-dose biological therapy; however, the risk of non-cutaneous cancers and melanomas combined trended towards significance with the unadjusted meta-analysis, the adjusted meta-analysis and the pooled method (p = 0.067, p = 0.060 and p = 0.110, respectively).

Time dependency of risk estimates

The duration of the individual clinical trials did not affect the OR for death, serious adverse events, non-cutaneous cancer and melanomas, or non-melanoma skin cancers (p = 0.751, p = 0.435, p = 0.760 and p = 0.993, respectively). The risk of serious infection with anti-TNF agents decreased significantly as the trial duration increased, p = 0.035 (fig 2). The estimated OR for serious infection decreased from 2.08 for trials of 12 weeks duration to 0.97 for trials of 104 weeks duration. The effect of time on the lymphoma risk could not be estimated due to the very high number of zero event studies.

• Download figure
• Open in new tab
• Download powerpoint

Figure 2

Regression of the logarithm of the odds ratio versus trial duration. Logarithm of the odds ratio of trials included in the analysis by trial duration, size of bubble is proportional in area to the trial’s weight in the analysis, shading of the circle related to dose used. β coefficient for duration is −0.00761 per week, p = 0.0512.

Heterogeneity and sensitivity analyses

Over 460 sensitivity analyses were performed. In only one case did a sensitivity analysis identify an increased risk when the base case did not (unadjusted meta-analysis of serious infection with high-dose adalimumab). Heterogeneity was evident in seven of the base case comparisons, when these analyses were performed with a random effects model the results were statistically different in two cases (serious infections with high-dose anti-TNF and infliximab). Table 6 reports the sensitivity analyses for all base case analyses that were heterogeneous or statistically significant.

DISCUSSION

In this systematic review, we combined trial data from 18 well-designed RCT, which to our knowledge represents the largest review of RCT for serious adverse events associated with anti-TNF therapy for RA. Several meta-analytic models were analysed. In general, adjusting the meta-analysis for exposure often decreased the risk estimates and test statistics, the reciprocal of the opposite treatment arm size continuity factor tended to produce higher risk estimates and test statistics than the 0.5 correction factor and the random effects model tended to produce estimates with wider confidence intervals. The simple pooling commonly used in product labelling identified statistically significant risks in three of the 126 individual risk assessments, whereas the meta-analytic techniques only identified two of these. The multiple comparisons created a 98.7% chance of at least one type one error. A common approach to minimising the chance of type one error with multiple comparisons is to adjust the significance criterion (for example from 0.05 to 0.01), the drawback being that the probability of a type two error is increased. We elected not to adjust the p value to maximise our chance of finding increased risks.

Our analysis showed that the risk of death, serious adverse events, serious infection, lymphoma, non-cutaneous cancer/melanoma or non-melanoma skin cancers was not increased with any anti-TNF at recommended doses. These results do not provide insight into the effects of long-term therapy as the included studies had a mean follow-up of less than one year. Although comparisons did not reach statistical significance many had very wide confidence intervals. The total number of subjects in the analysis was large; however, the sample was underpowered to detect differences for very rare events. For example, to detect a fourfold increase in the risk of lymphoma would require approximately 24 000 subjects. Finally, it should be noted that indirect comparisons have methodological drawbacks and do not possess the validity of randomised controlled trials and caution must be exercised when making any inferences.

Death

Observational studies in Spain and Sweden note decreased mortality with anti-TNF treatment with rate and hazard ratios of 0.32 (95% CI 0.20 to 0.53) and 0.65 (95% CI 0.46 to 0.93), respectively.48 49 Our analysis did not show decreased mortality with anti-TNF treatment. The mortality rate in the Swedish study was 0.016 per subject-year, which is more than twice the RCT rate of 0.006 per subject-year. This rate difference illustrates the effect of excluding subjects with co-morbidities in most RCT and may limit the generalisability of our results.

Serious infections

Our results are comparable to results from the National Data Bank for Rheumatic Diseases50 and the British Society for Rheumatology Biologics Register,51 which showed no increases in serious pneumonia or infections with anti-TNF treatment. In contrast, the Rheumatoid Arthritis Observation of Biologic Therapy registry reports a 2.7 to 2.8-fold increase in serious infections with etanercept and infliximab.52 Although registries provide important information, it is important to note that they may contain several known and unknown biases.53 Results from randomised controlled trials are useful in eliminating some of the biases inherent in registries; however, the careful subject selection and screening in randomised trials may lead to inaccurate estimations of real world safety.

Our study did not reproduce the increased risk of serious infection for anti-TNF treatment at recommended doses as reported in the meta-analysis by Bongartz et al.47 Bongartz et al47 report an OR of 1.8 (95% CI 1.1 to 3.1) for the comparison of low-dose anti-TNF antibodies using the meta-analytic approach, whereas our analysis produced an OR of 1.21 (95% CI 0.89 to 1.63; p = 0.24) for recommended doses of all anti-TNF using the unadjusted meta-analysis. This discrepancy may be explained by differences in study design, namely, the inclusion of etanercept, the addition of recent clinical trials, the use of only published data and different dose classifications.

Several observational registries50^–52 54 55 and pharmacoepidemiology analyses55 report that the risk of death, serious infection and malignancy is similar across anti-TNF. Within the context of treating RA patients, the similarities within the class seem to outweigh the differences between the individual agents and therefore we included etanercept in our analysis. We also included three additional studies of adalimumab and infliximab, all of which had fewer serious infections in the active groups than the recommended dose groups. Bongartz et al47 did not include the Maini et al41 publication in his analysis, whereas the trials of Abe et al,44 Breedveld et al34 and Westhovens et al45 were not available. The use of only published manuscripts resulted in different serious infection counts for the trials by Weinblatt et al,31 van de Putte et al33 and Westhovens et al.45 We reported one less infection in each trial than Bongartz et al.47 The use of FDA reports, conference abstracts and personal communications can provide additional information; however, they may create bias due to discordance in the methods of safety data reporting in integrated safety reports and published manuscripts,56 changes in information between conference presentations and the final peer reviewed publication57 58 and the lack of standardised information search strategies for these sources. The addition of previously available and newly published data along with a focus on published data resulted in lower overall risk estimates of serious infection for adalimumab and infliximab in our analysis.

Our analysis confirmed the dose-dependent increase in the risk of serious infection with anti-TNF treatment noted by Bongartz et al.47 We identified an increased risk of serious infection for all individual agents that were evaluated at high doses.

Dixon et al53 reported that in the clinical practice setting the risk of serious infection with anti-TNF agents peaks in the first 3 months. Our analysis confirmed that the risk decreases over time (p = 0.0351), therefore the duration of clinical trials must be considered when evaluating the risk of serious infection.

Malignancy

As expected, we noted considerable variation in the reporting of malignancies. Some malignancies that were reported in FDA reports did not appear in the original publications. Many manuscripts detailed specific malignancies in the body of the manuscript, whereas others aggregated malignancies either in the body of the manuscript or in a table. There was no consistency between trials on how malignancies were aggregated, for example, the study by Kremer et al59 combined all benign, malignant and unspecified neoplasms, whereas Westhovens et al45 aggregated non-melanoma skin cancers, benign neoplasms and carcinomas in situ. The published reports often failed to provide details on how malignancies were defined, recorded and reported. Our method of combining specific types of malignant events attempted to minimise bias by pooling like events; however, this significantly reduced the power of our analysis.

Our analysis did not provide evidence of an increase in the risk of lymphoma, non-melanoma skin cancers or the composite endpoint of non-cutaneous cancer plus melanoma for recommended doses or high doses of anti-TNF. Our analysis differed from the analysis by Bongartz et al47 because we grouped malignancies by type to avoid potential bias. In an ad hoc analysis of all abstracted malignancies (ie, lymphomas, skin cancers and non-cutaneous cancers) we calculated OR of 1.34 (95% CI 0.75 to 2.39) and 2.49 (95% CI 0.82 to 7.59) for recommended and high-dose anti-TNF, respectively. Of particular concern in our analysis was the fact that 15 malignancies in the anti-TNF therapy groups were unable to be allocated to the controlled or uncontrolled portion of the study, 14 of these were from the trial by Westhovens et al.45 These 14 malignancies were described as non-melanoma skin cancers, benign neoplasms and carcinomas in situ; they are not included in the meta-analyses but are included in the simple exposure-adjusted pooled analysis.

High doses

The norm in clinical practice is to start with recommended doses of anti-TNF and then increase the dose over time if required. In contrast, all of the high-dose treatment groups in our meta-analyses initiated therapy at the high dose. Vishalpura et al60 and Ollendorf et al61 estimated that 10–24% of subjects receiving adalimumab may double their dose over a one-year period. Stern and Wolfe62 reported that the mean infliximab dose in clinical practice increases to 0.64 mg/kg per week after 2 years of therapy. In our analysis, the mean adalimumab dose in the high-dose group was 49 mg/week or approximately 22% higher than the double dose group seen in clinical practice and 245% higher than the recommended dose. The high-dose infliximab group in our analysis received a mean dose of 1.16 mg/kg per week, which is approximately three times above the recommended dose and twice as high as the mean dose seen in clinical practice. Applying risk estimates from subjects who initiate therapy at high doses to subjects who “climb” to high doses over time may not be valid because the physiological processes that reduce relative efficacy may affect relative toxicity.63 As the high-dose groups in our analysis initiated therapy with a high dose and used doses higher than those normally seen in clinical practice, the extrapolation of our high-dose analyses to clinical practice may be inappropriate. The dose-dependent effects noted for adalimumab and infliximab could not be contextualised against etanercept due to a lack of non-biological controlled high-dose etanercept studies. Johnsen et al64 examined the safety of 25 mg and 50 mg of etanercept twice weekly in 77 subjects and noted an increased incidence of infectious adverse events with high etanercept doses.

Clinical significance

Our analysis of over 6000 subjects receiving recommended doses of adalimumab, etanercept or infliximab to treat RA over an average of 0.85 years did not identify an increased risk of important adverse events; however, there are many difficulties in precisely quantifying risk estimates for rare events. This uncertainty in quantifying the actual risks of anti-TNF therapy poses challenges for rheumatologists when informing patients about the risks of anti-TNF therapy. In an effort to provide practising rheumatologists with a tool to review quantitative data from RCT with patients we summarised the number of events in both the controlled and uncontrolled portions of the trial along with the appropriate risk estimate (table 7).