Jump to comment:

• Mischievous data mining of Registry Data
  Robert J Spiegel, M.D., FACP
  Published on: 24 June 2004

• Published on: 24 June 2004

  Mischievous data mining of Registry Data

  • Robert J Spiegel, M.D., FACP, Chief Medical Officer, Schering-Plough Research Institute

  Dear Editor

  We read with interest the recent article of Jonsdottir and colleagues,[1] in which the authors describe an increase in the rate of ACLA in RA patients who received any anti-TNF therapy; a potentially very interesting finding. However, this article commits multiple errors in scientific methodology and we are concerned that the presumed apparent correlations of ACLA are actually the result of inherent b...

  Show More

  Dear Editor

  We read with interest the recent article of Jonsdottir and colleagues,[1] in which the authors describe an increase in the rate of ACLA in RA patients who received any anti-TNF therapy; a potentially very interesting finding. However, this article commits multiple errors in scientific methodology and we are concerned that the presumed apparent correlations of ACLA are actually the result of inherent bias in study design, flawed analyses and an overinterpretation of a limited registry database. Of particular concern, without specification of the selection criteria, a subgroup of patients on TNF-á antagonists in STURE was analyzed. A small group of 121 patients has been extracted from a much larger registry experience - not to suggest a hypothesis generating observation – but to support a very far reaching headline and conclusion. The increase in the rate of ACLA in the reported patients who received any anti-TNF therapy appears to be real and quite interesting. However, following that observation, this article commits a variety of serious errors in scientific methodology and reporting. As of March 2004, the Swedish Rheumatology Registry had entered 2334 patients treated with biologics, including 1715 treated with infliximab.– it is not explained how the subgroup of 121 patients was selected. Only a subgroup of this subgroup, i.e. 44 of the 64 infliximab-treated patients, is then utilized to begin exploring the relationship of ACLA formation to other clinical outcomes, such as ACR response and infusion reactions. No explanation is provided as to why 20 of the 64 patients were not utilized in the analysis. No information is given regarding other known predictors of these outcomes, nor whether a proper analysis was done to determine if this was a major independent risk factor. What was done to adjust for baseline variables? Why wasn’t a proper regression analysis done? Why is there no disclaimer stating that the presence of ACLA cannot be assumed to have a causal relation to these outcomes based on these exploratory analyses?

  Of greater concern, comparative statements are made throughout the article regarding the comparable effects in etanercept-treated patients – but the data to support these statements are not shown! This is remarkable and it is hard to understand how an article could be published without such information being provided. How many of the 57 etanercept- treated patients had this information available? Why is their data not presented? How was the subset of 57 etanercept-treated patients selected from the much larger number in the registry? Importantly, there is no mention made that this registry does not randomize patients to infliximab vs. etanercept and no attempt is made to discuss their comparability. The presumed, not displayed, differences between infliximab and etanercept could have been caused by imbalances in baseline, and confounding variables between the infliximab-treated and etanercept-treated group. As just one observation, there is a great imbalance in the use of methotrexate at baseline in infliximab and etanercept groups (90% vs. 50%, respectively), possibly indicating differences in the disease severity of the two groups. Therefore, it is of particular concern that definitive statements are made throughout the article regarding the comparable effects in etanercept-treated patients – but the data to support these statements are not shown, nor do the baseline data shown support their comparability.

  A registry database can potentially provide certain data very well, e.g. large real world safety outcomes. It can also be a good source for hypothesis generating exploratory observations. However, registry data is rarely able to answer questions in the way a well designed randomized trial can. This report appears to represent the worst type of data dredging- a concern that makes people wary of the use of registry data for mischief making. In this case that concern is warranted.

  Referenc

  1. Jonsdottir T, Forslid J, Van Vollenhoven AM, Harju A, Brannemark SA, Klareskog L, et al. Treatment with TNF-{alpha} antagonists in patients with rheumatoid arthritis induces anticardiolipin antibodies (ACLA): ACLA predict worse clinical outcome with infliximab and more frequent treatment limiting infusion reactions. Ann Rheum Dis 2004.

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top