Background Pain management is central in the treatment of chronic rheumatic diseases. Pharmacological treatments are widely used, but many patients also seek non-pharmacological solutions to replace or complement pharmacological methods, due to residual pain or concern about side effects. Exercise is the primary non-pharmacological tool available. The possibility that reporting biases may influence results in exercise trials, however, is a concern. Reporting biases may result in the exaggeration of potential benefits if statistically significant or “positive” results are preferentially published compared to non-significant or “negative” results. Reporting bias can occur via (1) study publication bias, in which positive studies tend to be published, whereas negative studies are not; (2) selective outcome reporting bias, in which outcomes published are chosen based on statistical significance; and (3) selective analysis reporting bias, in which data are analyzed with multiple methods and only those that produce positive results are reported. Reporting biases may be particularly problematic in areas of research, such as exercise for pain management, where multiple self-report outcome measures are assessed and there is no agreed upon primary outcome in the field. Commonly-used statistical tools (e.g., funnel plots, asymmetry tests, regression methods) sometimes fail to detect these biases in this context, but Ioannidis' Test for Excess Significance (TES) may provide a more sensitive alternative.
Objectives The objective of this study was to demonstrate the use of the TES and test for evidence of reporting biases in an existing meta-analysis, which did not detect reporting biases using asymmetry-based regression methods.
Methods This was a secondary analysis of Kelley et al.'s 2011 meta-analysis of intention-to-treat randomized controlled trials (RCTs) that examined the effect of community-deliverable exercise interventions for decreasing pain in rheumatic diseases. To assess reporting bias, we applied the TES, which compares the observed number of statistically significant, or “positive”, results favoring exercise to the expected number based on the overall effect size and the combined statistical power of the trials.
Results Among 17 exercise versus control comparisons, 8 (47%) were positive, which was significantly greater than the expected number of positive results (4 trials, 24%, p=0.021) based on the meta-analytic summary effect estimate of g = -0.205.
Conclusions The TES may detect reporting biases in meta-analyses of non-pharmacological interventions in the rheumatic diseases, where more commonly used statistical approaches are less sensitive, particularly when included trials assess multiple self-report or “soft” outcomes.
Acknowledgements Mr. Levis' work was supported by a Canadian Institutes of Health Research, Institute of Musculoskeletal Health and Arthritis Studentship in Musculoskeletal Health and Arthritis (201401SMA). Dr. Thombs was supported by an Investigator Salary Award from the Arthritis Society. Dr. Kelley was partially funded by the National Institute of General Medical Sciences (NIGMS) of the National Institutes of Health (NIH) under award number U54GM104942. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Disclosure of Interest None declared