Materials and methods

To try to maximise the identification of research literature, three methods were employed. Firstly, searches were conducted on electronic literature databases to identify all articles related to the treatment of OA of the knee joint in humans. Ovid Medline (1965–3/98), Ovid Embase (1980–3/98), BIDS Institute for Scientific Information (1981–3/98), and the Cochrane Library were searched. All the searches were conducted during spring 1998. Box 1 shows the generic search strategy. Secondly, we hand searched the bibliographies of review articles for applicable studies. Thirdly, we conducted a postal questionnaire of the Osteoarthritis Research Society International (OARSI) to obtain unpublished, “grey” (published in newsletters, etc) or very recent research. Surveys were posted to 261 OARSI members world wide and we received 33 (13%) replies. This reduced our reliance on literature databases. At this point we also identified and removed duplicate articles. Owing to study constraints, we were only able to review articles published in the English language.

All relevant references were compiled on an Excel data sheet. Each article was located and then assessed according to the inclusion criteria. The inclusion criteria were any studies that examined interventions for OA of the knee in human patients. We were deliberately open with our selection criteria as we were interested in examining the entire research base, not just specific interventions or methodologies. Study selection was undertaken by JAC, with PAD reviewing a random 10% sample (selected using a random number table) to act as a reliability test (87% agreement).

All articles that met the inclusion criteria were copied and retained for further analysis. Data were collected by JAC using a standardised data extraction sheet. We collected data on study treatment; justification for study; study methodology; study statistics; study conclusion; funding source; and year of publication.

Results

We grouped studies into six intervention types (which covered the main treatment options) and four study methodology types. These groupings formed the basis of our analysis, and are listed below.

Treatment options:

• Alternative and complementary treatment—homoeopathy, folk medicine, transcutaneous electrical nerve stimulation (TENS), acupuncture, diet and supplements

• Oral drugs—NSAIDs, simple analgesics (paracetamol)

• Injected drugs—corticosteroids, hyaluronan

• Education and behavioural change—support, coping strategies, behavioural changes, lifestyle adaptations, dietary change and psychological treatments

• Exercise and physiotherapy—muscle strengthening, exercise regimens

• Surgery—total knee replacement, osteotomy, debridement.

Methodology:

• Trial—prospective experimental studies on human subjects

• Observational—retrospective or prospective cohort studies

• Review—general disease reviews, specific intervention reviews

• Meta-analysis—studies reporting pooled data from a number of clinical trials.

The combined search produced 5134 articles. After the inclusion criteria had been applied, duplication removed, and language constraints imposed, 930 articles were retained for analysis. (The full list of 930 articles is available on request from the authors.)

We examined how the research output had changed over time. Table 1shows the results for changes in research output by intervention type. Statistical analysis of trend in output showed that research into all interventions had significantly increased over the period 1950–94. The figures show a particularly rapid growth in the overall amount of research into OA of the knee joint between 1960 and 1994, with the situation beginning to stabilise after 1995. Pharmaceutical and surgical studies represented the majority of the literature, and have dominated the research output for many decades (59.1% drug, 25.6% surgical, 5.3% alternative and complementary, 6.5% physiotherapy and exercise, 3.6% educational). However, the most recent figures for 1995 onwards show a decline in the proportion of pharmaceutical (49.4%) and surgical (22.9%) research, and absolute numbers of articles for 1995 onward look unlikely to surpass those of the previous period (table 1). In contrast, over the same period alternative and complementary (7.6%), exercise and physiotherapy (14.1%), and educational and behavioural change (5.9%) all displayed growth in the overall percentage and absolute numbers of articles (table 1).

We investigated how study methodology had varied over the period of investigation (table 2). Statistical trend analysis shows that the use of all study methodology types, with the exception of observational studies, has increased significantly over the period 1950–95. The initial appearance and growth in the number of reviews and subsequently meta-analyses from 1980 onwards is clearly shown (table2).

Tables 3 and 4 summarise the findings of the review on treatments for OA of the knee joint. When the study outcomes were examined, a high percentage of all research supported the use of the intervention under investigation (94%). By “supported” we mean that authors supported the use of the agent under investigation in the abstract or conclusion of an article without necessarily giving statistically significant results. However, analysis shows that oral drug treatments are significantly more likely to provide a supportive result than studies of other intervention types (p<0.001 by χ² test). In contrast, surgical studies were significantly more likely to give a non-supportive result than other research (p=0.003 by χ²test). All other interventions showed non-significant results when tested for bias in reporting of outcome.

Table 4 presents a summary of reported funding sources (that is, where the source of funding was declared in the published article). Commercial funding was concentrated in the pharmaceutical sector (accounting for 89% of all reported commercially funded studies). Government and charity funding was more evenly spread across interventions, but was the most important reported funding source for exercise and physiotherapy (28% of total) and educational and behavioural change interventions (51.5% of total). However, a high proportion of all studies failed to report the source of project funding (78.5% of all studies). Analysis showed that projects that had been commercially funded were significantly more likely to support the intervention under investigation than non-commercially or non-specified funding source studies (p=0.024 by χ² test). We examined changes in reporting of funding, and found that disclosure of funding source increased significantly over the period 1950–95 (p=0.0027 by χ² test) from 13% during the '70s, to 16.6% during the '80s, and 28.8% during the '90s.

Discussion

Our discussion covers the three main issues to arise from the review. Firstly, that the research agenda does appear to mirror the wishes of research consumers. Secondly, some areas remain substantially underresearched and urgently need to be considered. Thirdly, overviews of entire subject areas need to be promoted.

The results of the review, unsurprisingly, show that research into OA of the knee joint has been dominated by pharmaceutical and surgical studies. However, the time series results (table 1) show that research output is changing over time; rising for new innovations (such as education) while stabilising or declining for more established interventions (such as NSAIDs). Furthermore, results show that these shifts in the research “agenda” for OA of the knee joint are in the same direction as calls for change by consumers (Tallon D,et al, unpublished data).10This is encouraging as it shows that the research agenda does mirror consumer wishes (even if not directly influenced by them), despite commercial, researcher, and publication biases. This interpretation may be an artefact rather than a true change: a result of literature databases having wider and more comprehensive coverage of publications for more recent years. But the steady growth in research output into non-pharmaceutical and non-surgical interventions (both in absolute and relative terms), suggests this is not the case.

We realise that this review would have benefited from a second independent observer for study selection and data extraction. It might also be argued that applying a quality assessment measure (for example, Chalmers, Jadad scales)11 ,12 would have been useful, but a standard assessment for different study types has not been available until recently.13 Also, research suggests that the use of quality assessment instruments may be limited.14 ,15 We were only able to examine English language articles, and studies have shown that this may introduce bias into reviews.16Furthermore, we appreciate that the excessive use of literature databases may be problematic. The increase in research output we have seen may in part be due to databases having incomplete temporal coverage (for example, they do not include all back issues of journals). Also, our search strategy may not have been sensitive enough to identify all relevant articles. So we cannot claim the research base that we have identified represents the entire literature base. However, we made our search strategy deliberately open, avoiding specific intervention terms, as we wanted to avoid biasing results towards any one intervention or relying too much on the database tools to select relevant research.

It is already known that the research base is influenced by a number of factors, and such biases have been shown by previous research.8 ,9 So we have not concentrated our discussion on these areas. But our results confirm previous findings that factors such as commercial funding influence the published research base.

Having highlighted some encouraging trends in the research agenda, we can nevertheless make some clear recommendations for further research based upon our results. We have identified three main areas that appear to be underresearched; certainly underrepresented in our search results. Firstly, few studies have been undertaken comparing different interventions—for example, conservative treatment compared with surgery. A few examples do exist,17-20 but this area warrants further attention. Secondly, OA of the knee joint, like many other chronic conditions, requires long term management. It is unlikely that any one treatment will be effective at all times. It is therefore necessary to know more about the progression of the disease and which interventions are most effective at different stages for individual patients. Thirdly, it is evident that for the goal of more comparative evidence to be achieved, more evidence from randomised controlled trials (RCTs) needs to be collected for all interventions, with the exception of drugs where the RCT method is already standard practice. Alternatively, reviewers may have to start widening their inclusion criteria to allow high quality observational data to be included. If reviews, and Cochrane reviews in particular, are to reflect the reality of patient treatment and preferences then this problem must be considered.

As we have shown, the research agenda does appear to mirror consumer wishes (at least for literature on interventions for OA of the knee joint). However, several areas still require further evaluation. The areas we have flagged as underresearched may be difficult to research, which might explain why they have yet to be investigated. However, these areas are not impossible to research. For example, the continuing existence of surgical waiting lists in the United Kingdom means that surgery can be compared with other interventions, with people being randomly allocated to alternative treatments while on the waiting list for surgery.

Our final point relates to methodology. We feel that the type of overview of the literature conducted here, coupled with consumer surveys, will become increasingly important for the identification of areas of underresearch. As medical science advances and fragments it will be necessary to retain some form of overview of the research base to ensure that evidence based medicine is representative of consumer needs.