Tribute to alvan r. feinstein
Epidemiologic methods: the “art” in the state of the art

https://doi.org/10.1016/S0895-4356(02)00529-2Get rights and content

Abstract

When performing empirical research in public health and medicine, the investigator is typically faced with a variety of methodologic issues to resolve at the design and analysis stages of the research. The investigator must specify the research question, conceptualize and operationalize the variables to be measured, consider several research designs to choose from, determine appropriate measures of disease frequency and effect, address potential biases, devise the analytic strategies to follow, choose the appropriate software procedures or packages to use, carry out the analysis, and interpret the results. Most of these issues concern principles and methods of epidemiology and biostatistics, which, taken together, embody the methodologic “science” that underlies such research. Nevertheless, in bringing all these issues together to achieve a coherent and valid research conclusion, there is an “art” that is required that goes beyond the quantitative mechanics involved in carrying out the research. The art part is not easily quantifiable and typically is more skillfully performed as one becomes a more experienced investigator. Nevertheless, such methodologic art can be addressed methodologically through guidelines that identify the options or strategies available and suggest how appropriate choices can be made from such options or strategies. Selection bias and mathematical modeling are especially addressed.

Introduction

Over 20 years ago, as a young Assistant Professor in Biostatistics at the University of North Carolina at Chapel Hill, I was fortunate enough to have the opportunity for a personal meeting with Nathan Mantel, one of the giants in the fields of statistics and epidemiologic methods, who recently passed away earlier this year. At the time of my meeting with Dr. Mantel, I was in the process of writing the textbook “Epidemiologic Research: Principles and Quantitative Methods” with co-authors Lawrence L. Kupper and Hal Morgenstern [1]. We had just finished writing a chapter on the logistic model and how it could be applied to analyze epidemiologic data. We also wanted to consider adding a chapter that addressed several of the more subtler aspects of modeling, concerning the fundamental issue of how to determine a “best” model when there were so many possible strategies and algorithms from which to choose. Dr. Mantel gave us a very concise answer to our basic question: “It's an art!” he said and didn't attempt to add much more detail to the discussion.

It was clear to me that with his particular genius and experience in the fields of biostatistics and epidemiology over many years, Dr. Mantel was as much an artist as anyone regarding the “best model” and likely many other issues. However, his answer was not satisfying because it left the rest of us at a disadvantage of not having any guidelines on how to proceed or how to critique the mathematical modeling work of others. For these reasons, we decided that it was worthwhile to attempt to put more science into the art, with the hope that the art part might be improved and communicated to a general audience. The immediate result was a chapter in our Epidemiologic Research text entitled “Modeling: Analysis Strategy,” which has been the basis for a continually evolving point of view on how to determine the “best model” that I have been teaching and writing about for many years.

The above anecdote concerning the best model issue reflects a much broader concern, which is the focus of this article: “How do we address and improve the ‘art’ part in the state of the art of any/all methodologic research activity?” In general, when embarking on a research project, we face the task of bringing together all of the concepts, designs, analytic procedures, and even software into a coherent research strategy and implementation. Many issues arise during the research process that cannot be resolved by strictly following a set of rules or formulae. At best, the answer to many issues is often “it depends,” an expression that indicates the subjective nature of such an issue but that also suggests a need for specifying those options or guidelines that can lead us to good choices. Even though there are no easy answers to improving methodologic art, the quality of one's research can benefit from further developing the art and from developing new methods.

Section snippets

Methodologic art issues

The following is a non-exhaustive list of issues that I categorize under the rubric of methodologic art issues. This is a list of recurring methodologic problems derived from my teaching and research experience that always seem to have an “it depends” aspect of any attempt at resolution. (The list is not in order of importance.)

  • •

    A. What study design is “best”?

To illustrate why art is needed and to describe how guidelines, rather than rules, are the best that one can hope for, we focus on two of

Summary and conclusion

Methodology is not just a mechanical process. It requires insight, experience, and judgment. The investigator in public health and medical research must incorporate such “art” into the science of one's investigation. A (non-exhaustive) list of “methodologic art” issues has been outlined. Two such issues have been addressed in detail—assessing selection bias and determining a “best model.” The purpose here has been to illustrate what kind of methodologic issues require art to be performed (an

References (11)

  • D.G. Kleinbaum et al.

    Epidemiologic researchprinciples and quantitative methods

    (1982)
  • O.S. Miettinen

    Estimability and estimation in case-referent studies

    Am J Epidemiol

    (1976)
  • D.F. Ransohoff et al.

    Problems of spectrum and bias in evaluating the efficacy of diagnostic tests

    New Engl J Med

    (1978)
  • J.A. Knottnerus et al.

    The influence of referral patterns on the characteristics of diagnostic tests

    J Clin Epidemiol

    (1992)
  • D.G. Kleinbaum

    ActivEpi

    (2002)
There are more references available in the full text version of this article.

Cited by (26)

  • Lower back pain in golfers: a review of the literature

    2005, Journal of Chiropractic Medicine
View all citing articles on Scopus

Note: This paper is dedicated to the memory of Alvan Feinstein. Dr. Feinstein made many creative contributions to the fields of biostatistics and epidemiology over a long and storied career. I believe that his writings and presentations reflect a thoughtful appreciation and awareness of the importance of the “art” issues involved in any scientific inquiry.

View full text