Elsevier

Annals of Emergency Medicine

Volume 48, Issue 6, December 2006, Pages 750-756.e21
Annals of Emergency Medicine

How scientific journals work/original research
From Submission to Publication: A Retrospective Review of the Tables and Figures in a Cohort of Randomized Controlled Trials Submitted to the British Medical Journal

Presented at the 5th International Congress on Peer Review and Biomedical Publication, Chicago, IL, September 16-September 18, 2005.
https://doi.org/10.1016/j.annemergmed.2006.06.017Get rights and content

Study objective

We characterize the quantity and quality of data tables and figures in reports of randomized controlled trials (RCTs) submitted to the British Medical Journal (BMJ) and published in peer-reviewed journals. We investigate how the peer review process affected table and figure quality.

Methods

We reviewed 62 consecutive reports of RCTs submitted to the BMJ in 2001 that were later published in the BMJ (n=12) or elsewhere. We counted and categorized the tables and figures in both the initial submissions and published articles. Using standardized instruments and procedures, we analyzed the quality of these tables and figures and checked BMJ editorial documents to see whether changes were triggered by their review process.

Results

The numbers of tables and figures did not change markedly between submission and publication. Five percent of publications had no data tables; 56% had no data figures. Data density was low for published tables and figures. Tables seldom showed data stratified on important covariates; 88% of published tables were simple lists or were stratified on only 1 variable. Less than half the figures met their data presentation potential, with most failing to portray by-subject data and few displaying advanced features such as pairing, symbolic dimensionality, or small multiples. BMJ external peer reviewers seldom commented on tables or figures.

Conclusion

Tables and figures can convey details and complex relationships not easily described in text. Although tables are included in most submitted and published articles, they are not presented optimally; figures are used sparingly and are also of suboptimal quality. Journals should consider improving their table and figure quality in the hope that improved graphics will empower readers to scrutinize the data, thereby dissuading authors from presenting biased analyses and misrepresented conclusions.

Introduction

Tables and figures are a powerful means of conveying investigational results.1, 2, 3, 4, 5, 6 When properly constructed, they succinctly convey complex relationships prose cannot describe. Tables and figures bring readers closer to the actual data than prose, and their use can reduce the potential for biases that can occur when the dimensionality of data is reduced.7 Dimensionality is reduced when the investigator presents summaries of the data rather than the data points themselves. For example, an investigator who collected the values of 2 continuous variables on 100 patients could present them as a scatterplot (200 numbers), 2 box plots (10 numbers+any outlying values), or a bar graph of each variable’s mean (2 numbers [+4 more if confidence intervals (CIs) are presented]). Only the scatterplot preserves the dimensionality of the data and conveys the relationship between the 2 variables within each patient that is inherent to this data set.

With each decrease in dimensionality, there is the potential that truth and nuance will be masked.

Given the potential importance of tables and figures, one might expect that the medical journal peer-review process would pay particular attention to them. Preliminary results suggest otherwise.8 Day et al found that only 6% of general reviewer comments concerned tables or figures (D. Schriger, written communication, August 2002). Cooper et al9, 10 found that many figures in large- and medium-circulation journals were suboptimal. Although there has been little empirical research on the qualities that make for a good table or figure, there has been a fair amount of theoretical work. This work suggests that tables and figures should be self-explanatory, should show overall patterns and detailed information, and should have a data density—the amount of information presented per cm2—that justifies their existence (ie, are denser than text).1, 2, 3, 4, 5, 6, 7

We executed this study to assess formally the quality of tables and figures in reports of randomized controlled trials (RCTs) submitted to a general medical journal and to examine how often and in what ways they were modified between submission and publication. We chose to study the figures and tables in reports on a single study design, the RCT, for 2 reasons. First, the kinds of tables and figures required to present data from studies of similar design should be few and would be expected to be somewhat consistent among articles. Second, much of the content of reports of RCTs has been standardized in recent years because of CONSORT, and we wanted to see whether the CONSORT statement, though it makes no recommendations about data tables and data figures, might nevertheless have influenced the content of tables and figures.11, 12 In short, we wanted to understand how tables and figures were being used in reports of RCTs, whether tables and figures met their potential as a powerful means of conveying study results, and whether the peer-review process improved the quality of published tables and figures.

Section snippets

Materials and methods

We identified 72 consecutive reports of RCTs submitted to the British Medical Journal (BMJ) in 2001, 12 of which were subsequently published in that journal. By searching MEDLINE and Google by article title and author names, we identified an additional 50 articles that were published in specialty journals after rejection by BMJ; none were published in another general medical journal. We presume that the 10 remaining submissions have not yet been published as of September 2006. We obtained and

Results

The 62 submissions that have been published composed our study sample. Records of the editorial process were available for 57 articles (5 records could not be found). Thirty-five of these articles had been formally reviewed by external peer reviewers or BMJ statisticians. Five (14%) of the 35 external reviews contained comments about the article’s figures, and 16 (46%) had comments about the tables.

Limitations

There is no established, validated standard for graphic and tabular excellence. We believe that scientific articles should present sufficient data so that the reader can fully evaluate the information and reach his or her own conclusions about the results. Ziman16 called this “consensibility” and stressed that it was a necessary condition for the scientific process. The judgments we made in this article are consonant with this belief; we would be remiss, however, if we failed to acknowledge the

Discussion

Figures can convey complex information and can be used to tell the story of an investigation. Detailed tables, especially when stratified on variables of interest, can do the same. Yet only 42% of this cohort of reports of RCTs submitted to BMJ in 2001 had any data figures, and the editorial process at BMJ and other journals that refereed these articles increased this number by only 2%. Gelman et al17 argue persuasively that figures should replace tables in many circumstances, but we saw little

References (24)

  • H. Wainer

    Visual Revelations: Graphical Tales of Fate and Deception from Napoleon Bonaparte to Ross Perot

    (2000)
  • H. Wainer

    Graphic Discovery: A Trout in the Milk and Other Visual Adventures

    (2004)
  • Cited by (41)

    • Reporting results in manual therapy clinical trials: A need for improvement

      2021, International Journal of Osteopathic Medicine
    • Graphic Portrayal of Studies With Paired Data: A Tutorial

      2018, Annals of Emergency Medicine
      Citation Excerpt :

      The columns indicate the type of graph: bar, box, or histogram. The number below each graph is the number of unique pieces of information provided in the graphic.6,7 It should be apparent that bar graphs, whether they include a variability measure (bottom graph) or not (top graph), convey almost no information.

    • Graph Quality in Top Medical Journals

      2017, Annals of Emergency Medicine
      Citation Excerpt :

      Research on medical journals’ instructions for authors indicates that only 13% of journals provide any instructions on the use of graphs.13 Another study at a single journal found that peer reviewers rarely commented on graphs, and the few changes in graphs that originated at the editorial level were generally cosmetic.12 More than a decade ago, we analyzed graphs in Annals of Emergency Medicine, in the Journal of the American Medical Association (JAMA), and in pharmaceutical advertisements.6-8

    • The Effect of Peer Review on the Quality of Data Graphs in Annals of Emergency Medicine

      2017, Annals of Emergency Medicine
      Citation Excerpt :

      Each point in a scatter plot received 3 points, 1 for each axis value and 1 for the linkage of those 2 values. In all graphs, points were awarded for additional unique information such as the quantity depicted by an axis, labels, annotations, regression lines, and statistics (see Schriger et al5 and Tufte14 for details). For example, the data density index numerator for Figure 1, a bar graph, was calculated as 18 bars plus 18 bar identifiers plus 1 for the labeled, distance-dependent x axis, or 37 points in total.

    • Graphs, tables, and figures in scientific publications: The good, the bad, and how not to be the latter

      2012, Journal of Hand Surgery
      Citation Excerpt :

      For scientists who wish to communicate their results with a large body of readers, it is crucial to design graphics that maintain “accuracy in [their] data and clarity in [their] presentation.”11 Doing so will enable readers to glean important findings and relationships quickly and easily, and reduce the risk of misinterpretation.2,3,5,6,8 Graphs and tables also take up less space and fewer words than text that explains the same information, making them valuable techniques for presented data.3

    View all citing articles on Scopus

    Supervising editor: Michael L. Callaham, MD

    Author contributions: DLS and DGA conceived the study and laid out the initial design. SS obtained the research material and helped with the design. DLS designed the data forms; these were reviewed by all authors and pilot tested by RS and PYL. DLS and RS did the data abstraction for the figures study. DLS, RS, and PYL did the data abstraction for the tables study. DLS did the data analysis and drafted the article. RS drafted Appendix E4. All authors actively participated in the editing of the article and approved the final version. DLS takes responsibility for the paper as a whole.

    Funding and support: Ms. Sinha was funded in part by a summer research grant from the UCLA School of Medicine Short Term Training Program.

    Publication dates: Available online September 15, 2006.

    Reprints not available from the authors.

    View full text