Article Text

Download PDFPDF

A survey of inclusion of the time element when reporting adverse effects in randomised controlled trials of cyclo-oxygenase-2 and tumour necrosis factor α inhibitors
  1. Y Yazici1,
  2. H Yazici2
  1. 1New York University, Hospital for Joint Diseases, New York, New York, USA
  2. 2Department of Medicine and Division of Rheumatology, University of Istanbul, Istanbul, Turkey
  1. Correspondence to:
    Dr Y Yazici
    515 East, 72nd Street, Number 29E, New York, NY 10021, USA; yusuf.yazici{at}nyumc.org

Abstract

Background: The adequacy of reporting the time element in adverse effects in articles on randomised clinical trials of cyclo-oxygenase-2 and tumour necrosis factor (TNF)α antagonists was surveyed.

Methods: Prominent rheumatology and general/internal medicine journals were searched for all randomised controlled trials published about cyclo-oxygenase-2 and TNFα inhibitor use in rheumatological diseases up to November 2005. Reporting of time to the occurrence of the adverse effects, the use of patient years as the time frame of the reported adverse effects and the use of annual standard incidence ratios based on the surveillance, epidemiology and end-results (SEER) programme when reporting neoplasms as potential adverse effects of TNFα antagonists were specifically tabulated.

Results: Only 23 of 70 (33%) of all articles gave the specific time of onset of an adverse effect. Nine studies used patient years to report the adverse effects and six studies used annual standard incidence ratios, using SEER, as the comparator.

Conclusion: In reporting of adverse effects in randomised clinical trials, a particularly neglected issue is the reporting of the time dimension of adverse effects.

  • COX-2, cyclo-oxygenase-2
  • RCT, randomised controlled trial
  • SAE, serious/severe adverse event
  • SEER, surveillance, epidemiology and end-results
  • TNF, tumour necrosis factor

Statistics from Altmetric.com

There is recent medical and public concern about the adverse effects of cyclo-oxygenase-2 (COX-2) and tumour necrosis factor (TNF)α inhibitors. There have been and continue to be methodological problems when reporting adverse effects in clinical trials in general, including those in rheumatology.1,2 We suggest that a particularly overlooked methodological issue has been the necessary importance that should be given to the time element when reporting adverse effects. We have already voiced this concern as it is related to the time of onset of lymphomas, tuberculosis and demyelinating central nervous system disease in association with TNFα antagonist use.3,4

The purpose of this work was to formally survey the time dimension of adverse effects in clinical trial reporting, as it has related to COX-2 and TNFα antagonists, in the main rheumatology and general medicine journals. We are unaware of a similar survey.

METHODS

Using PubMed, we searched five prominent rheumatology (Arthritis and Rheumatism, Annals of the Rheumatic Diseases, Journal of Rheumatology, Rheumatology and Clinical and Experimental Rheumatology) and six medicine journals (New England Journal of Medicine, Lancet, Annals of Internal Medicine, Archives of Internal Medicine, American Journal of Medicine and Journal of American Medical Association) for randomised controlled trials (RCTs) of COX-2 and TNFα inhibitor use in rheumatological diseases before November 2005. The terms “etanercept”, “infliximab”, “adalimumab”, “celecoxib, “rofecoxib” and “valdecoxib” were searched separately with the limits of “randomized controlled trial”, “English” and “human” set as defaults. Studies where the primary outcome was to compare different imaging outcomes, examining only quality of life, biopsy/tissue changes and pharmacokinetics of already reported trials, RCTs in paediatric populations, orthopaedic, dental, pain-management studies and review articles containing our search terms were excluded, as well as any subanalysis reports.

YY searched through each article using a standard checklist. Three items were sought: (1) whether time of onset of an adverse effect relative to the time the study drug started was reported; (2) whether surveillance, epidemiology and end-results (SEER) was used in calculating the annual standard incidence ratio as a comparator in listing or discussing the malignancies; and (3) whether patient years was used as the time frame for adverse effects.

All adverse effects and serious adverse effects and serious/severe adverse events (SAEs) were tabulated separately. An adverse effect was recorded as an SAE only if it was classified specifically as such in the RCT, in text, in a table or as a Kaplan–Meier curve. Laboratory abnormalities, if considered as an adverse effect or an SAE, again by the RCT, were included as well.

RESULTS

Seventy articles were identified; 64 (91%) were industry sponsored; 66 were from North America or Europe (appendix 1). Table 1 shows that 23/70 (33%) of the articles gave the specific time of onset of any adverse effect in text, in a table or as a Kaplan–Meier curve.

Table 1

 Characteristics of and time to adverse effect reporting in randomised controlled trials of cyclo-oxygenase-2 and tumour necrosis factor α inhibitors

Among articles on TNF inhibitors, 17 of 44 (39%) studies gave the time to adverse effect. In all except one, time to event was given for SAEs only. In seven of these, time to SAEs was available for all SAEs. The remaining nine studies gave time to SAEs in ⩽50% of the patients. Of these, one study gave only the mean time to SAEs, whereas another mentioned the time to event only for malignancies.

A total of 6 of 26 (23%) studies on COX-2 inhibitors gave time to adverse effect. Three reported SAEs; all gave time to event for <50% of the SAEs reported. Three studies reported this as Kaplan–Meier curves, one of these reported SAEs with no specific time information provided. Curves did not distinguish between adverse effects and SAEs in any of the three studies.

Two COX-2 and four TNF inhibitor trials used patient years for reporting adverse effect rates.

Seventeen trials on rheumatoid arthritis (patients, n = 7197) with TNF inhibitors reported one or more than one malignancy, and in 8 of 17 studies the data from SEER, based on annual incidences, were used for comparisons.

DISCUSSION

Among the RCTs we surveyed, the adequacy of time to adverse effect reporting was wanting. In more than two thirds of the studies the time element of adverse effect was not considered and this was also true for SAEs.

The time to event issue had also not been part of the revised CONSORT.5 However, it was brought up in the recent updating of CONSORT related to adverse effects,2 where the authors said “…specifying the timing of events may be important”.

We need to know more. Clustering of adverse effects in time, whether early after drug use—as is usually the case6—or late, is an important clue to causality.7 For example, the prescription-event monitoring data for meloxicam from the UK show that incidence densities for dyspepsia and nausea/vomiting are twice as common in the first 2 months after treatment initiation compared with those after 2–6 months.7 The instantaneous risk for different drugs can be very different from each other even though they all might have the same average risk which, for the total period of follow-up does not represent the real risk faced by the patients.8 We propose that RCT reporting should include timing of adverse effects or at least declare that a formal analysis was done and no clustering or skewing with respect to time observed. Finally, if the sample size is not adequate for a formal statistical analysis, it can always be said that a trend was or was not present, with the presence of a trend necessitating display of hard data.

Authors of RCTs sometimes resort to the concept of “patient years” to define the time frame of adverse effect incidence. Six such manuscripts were found in the current survey. However, as reported previously,2,9,10 this practice has problems. For example, the relatively rare idiosyncratic drug reactions usually occur early in the treatment course and in only a few people.11 Apart from the few patients with adverse effects, the remaining patients who are prescribed the drug will never have these reactions however long they use the drug. This will unduly inflate the denominator of the related incidence ratio and thus will have the potential of under-representing the problem related to this particular adverse effect. Similarly, Ioannidis et al,2 in their recent paper, point out that late-onset adverse effects are also apt to be missed when we use the patient years. In short, only “…when an event is (or is believed to be) likely to occur at any stage during continuous treatment with a drug then an event rate with a time component (rate per person year, etc) has a true meaning.”9

Another important issue in using the patient years as the time frame is whether more than one event per patient goes into the numerator. This needs to be clearly explained in the methods. If more than one adverse effect/patient is included in the numerator, the statistics derived from that incidence ratio will be erroneous.9 An adverse effect can repeat itself, like skin rashes in TNFα antagonist use in any one patient. This can lead to over-representation of the said individual in tests of significance, which in turn have their arithmetic basis on non-interdependence of the observations they test for non-randomness.

The less than adequate attention given to “time of onset” of an adverse effect is also apparent in the authors’ usage of standardised incidence ratios when reporting malignancies. The usual comparator here has been the SEER programme.12 SEER tables give annual incidence rates for malignancies in the general population, which are expected to be evenly distributed through the year. On the other hand, if there is, as we discuss above, some skewing as to the time of such malignancies in the treatment arm, this would make comparisons based on an annual incidence inaccurate. This was the issue we had voiced as it related to the concern of lymphomas with TNFα antagonist use.3 In the current survey, there were eight articles that used the SEER database as a comparator and no indication that an attempt had been made to account for possible skewing of these events with respect to time in the treatment arm.

There are guidelines to help us with adverse effect reporting.2 Their widespread use should be endorsed,13 although, on the basis of our findings in the current survey, we should pay more attention to the time to adverse effect issue.

RCTs are usually limited in their follow-up time, and late and rare adverse effects may be missed. Long-term, real-world follow-up studies are needed to realistically assess the full safety picture for any given treatment.14 Added to this, most RCTs do not have enough power to detect significant differences for overall effects. One solution would be to compile data from many RCTs, but that has not been done often.15

We suggest that a more meticulous reporting of adverse effects in RCTs will not only help us as care givers to understand adverse effects better in their biological and clinical context but will also more precisely inform the care takers about the important issue of adverse effects.

REFERENCES

APPENDIX

Footnotes

  • Competing interests: None declared.

  • Published Online First 10 July 2006

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.