Epidemiology of research into interventions for the treatment of osteoarthritis of the knee joint
- Mr Jiri Chard, MRC—Health Services Research Collaboration, Department of Social Medicine, University of Bristol, Whiteladies Road, Bristol BS8 2PR, UK Email:
- Accepted 10 January 2000
OBJECTIVE To assess the published research base for interventions for osteoarthritis of the knee, and to identify areas in need of further research.
METHODS Literature searches were conducted on electronic databases (Medline, Embase, ISI, and Cochrane library), bibliographies of existing review articles were hand searched, and a postal questionnaire was sent to members of the Osteoarthritis Research Society International. All relevant articles were copied and searched for treatment type, study methodology, statistical results, conclusions, funding source, researcher affiliations, and year of publication, using a predetermined data extraction form.
RESULTS There have been marked changes in the literature over the period studied (1950–98), with a recent rise in trials of physical therapy, educational interventions, and complementary treatments. However, overall, most research was either drug (59.1%) or surgically (25.6%) related. Most of the studies reported positive results (94%). Research on oral drugs was significantly more likely to provide a positive result than research on any other intervention (p<0.001 by χ2 test). Commercially funded studies were significantly more likely to produce a positive result than non-commercially funded research (p=0.0027 by χ2 test).
CONCLUSIONS Analysis of time trends indicates that the research agenda does shadow changes in consumer demands. However, there are significant gaps in the research base that need to be considered.
The amount of research being undertaken into osteoarthritis (OA) has grown steadily over the past 50 years. This research effort has been both basic and applied, with a renewed interest in the pathogenesis of OA, and in its treatment. However, the quantity of available evidence has become too large for any individual practitioner to handle. As a solution, systematic reviews, meta-analyses, and guidelines have been published to summarise the research base, so that practitioners can more easily assimilate the research, and provide treatment based on the best evidence available. A number of guidelines are available for the treatment of OA.1-3 In addition, the Cochrane Collaboration Musculoskeletal Review Group has produced several reviews on various aspects of OA treatment (specifically the use of non-steroidal anti-inflammatory drugs (NSAIDs), knee replacement, and complementary treatment).4-6
Currently, most reviews use standard literature search techniques to identify research articles in a specific area of interest (usually a certain intervention, disease, or methodology). They then classify the primary research by examining the study methodologies, and provide recommendations on future treatment based on the research. This methodological approach has greatly enhanced the use of review articles, compared with previous non-systematic, non-specific reviews. Summaries of the best evidence are now available to practitioners for use in their work. However, systematic reviews are not perfect, and still contain many forms of bias—for example, publication bias,7 commercial funding bias,8 9 and researcher interest bias. Another concern about systematic reviews is that they may be too specific. More generalised overview reviews can be helpful in identifying areas that have been underresearched.
This review aimed at examining the “epidemiology” of the research base for interventions used in the treatment of OA of the knee joint, and to make recommendations for future areas of research based on these findings. By epidemiology we mean examining the structure of the research base (overall structure, classification, and changes in output over time). We were not conducting a meta-analysis, or a systematic review on a specific intervention or methodology.
Materials and methods
To try to maximise the identification of research literature, three methods were employed. Firstly, searches were conducted on electronic literature databases to identify all articles related to the treatment of OA of the knee joint in humans. Ovid Medline (1965–3/98), Ovid Embase (1980–3/98), BIDS Institute for Scientific Information (1981–3/98), and the Cochrane Library were searched. All the searches were conducted during spring 1998. Box 1 shows the generic search strategy. Secondly, we hand searched the bibliographies of review articles for applicable studies. Thirdly, we conducted a postal questionnaire of the Osteoarthritis Research Society International (OARSI) to obtain unpublished, “grey” (published in newsletters, etc) or very recent research. Surveys were posted to 261 OARSI members world wide and we received 33 (13%) replies. This reduced our reliance on literature databases. At this point we also identified and removed duplicate articles. Owing to study constraints, we were only able to review articles published in the English language.
Box 1 Generic search strategy
Osteoarthritis or synonyms as keyword and/or MeSH heading search
Knee as keyword and/or MeSH heading search
(Survey or synonyms as keyword and/or MeSH heading search
Experiment as keyword and/or MeSH heading search
Clinical trials or synonyms as keyword and/or MeSH heading search
Observational/cohort studies and synonyms as keyword and/or MeSH heading search
Review as keyword and/or MeSH heading search
Systematic reviews or synonyms as keyword and/or MeSH heading search
Meta-analysis as keyword and/or MeSH heading search
Management guidelines as keyword and/or MeSH heading search)
All relevant references were compiled on an Excel data sheet. Each article was located and then assessed according to the inclusion criteria. The inclusion criteria were any studies that examined interventions for OA of the knee in human patients. We were deliberately open with our selection criteria as we were interested in examining the entire research base, not just specific interventions or methodologies. Study selection was undertaken by JAC, with PAD reviewing a random 10% sample (selected using a random number table) to act as a reliability test (87% agreement).
All articles that met the inclusion criteria were copied and retained for further analysis. Data were collected by JAC using a standardised data extraction sheet. We collected data on study treatment; justification for study; study methodology; study statistics; study conclusion; funding source; and year of publication.
We grouped studies into six intervention types (which covered the main treatment options) and four study methodology types. These groupings formed the basis of our analysis, and are listed below.
Alternative and complementary treatment—homoeopathy, folk medicine, transcutaneous electrical nerve stimulation (TENS), acupuncture, diet and supplements
Oral drugs—NSAIDs, simple analgesics (paracetamol)
Injected drugs—corticosteroids, hyaluronan
Education and behavioural change—support, coping strategies, behavioural changes, lifestyle adaptations, dietary change and psychological treatments
Exercise and physiotherapy—muscle strengthening, exercise regimens
Surgery—total knee replacement, osteotomy, debridement.
Trial—prospective experimental studies on human subjects
Observational—retrospective or prospective cohort studies
Review—general disease reviews, specific intervention reviews
Meta-analysis—studies reporting pooled data from a number of clinical trials.
The combined search produced 5134 articles. After the inclusion criteria had been applied, duplication removed, and language constraints imposed, 930 articles were retained for analysis. (The full list of 930 articles is available on request from the authors.)
We examined how the research output had changed over time. Table 1shows the results for changes in research output by intervention type. Statistical analysis of trend in output showed that research into all interventions had significantly increased over the period 1950–94. The figures show a particularly rapid growth in the overall amount of research into OA of the knee joint between 1960 and 1994, with the situation beginning to stabilise after 1995. Pharmaceutical and surgical studies represented the majority of the literature, and have dominated the research output for many decades (59.1% drug, 25.6% surgical, 5.3% alternative and complementary, 6.5% physiotherapy and exercise, 3.6% educational). However, the most recent figures for 1995 onwards show a decline in the proportion of pharmaceutical (49.4%) and surgical (22.9%) research, and absolute numbers of articles for 1995 onward look unlikely to surpass those of the previous period (table 1). In contrast, over the same period alternative and complementary (7.6%), exercise and physiotherapy (14.1%), and educational and behavioural change (5.9%) all displayed growth in the overall percentage and absolute numbers of articles (table 1).
We investigated how study methodology had varied over the period of investigation (table 2). Statistical trend analysis shows that the use of all study methodology types, with the exception of observational studies, has increased significantly over the period 1950–95. The initial appearance and growth in the number of reviews and subsequently meta-analyses from 1980 onwards is clearly shown (table2).
Tables 3 and 4 summarise the findings of the review on treatments for OA of the knee joint. When the study outcomes were examined, a high percentage of all research supported the use of the intervention under investigation (94%). By “supported” we mean that authors supported the use of the agent under investigation in the abstract or conclusion of an article without necessarily giving statistically significant results. However, analysis shows that oral drug treatments are significantly more likely to provide a supportive result than studies of other intervention types (p<0.001 by χ2 test). In contrast, surgical studies were significantly more likely to give a non-supportive result than other research (p=0.003 by χ2test). All other interventions showed non-significant results when tested for bias in reporting of outcome.
Table 4 presents a summary of reported funding sources (that is, where the source of funding was declared in the published article). Commercial funding was concentrated in the pharmaceutical sector (accounting for 89% of all reported commercially funded studies). Government and charity funding was more evenly spread across interventions, but was the most important reported funding source for exercise and physiotherapy (28% of total) and educational and behavioural change interventions (51.5% of total). However, a high proportion of all studies failed to report the source of project funding (78.5% of all studies). Analysis showed that projects that had been commercially funded were significantly more likely to support the intervention under investigation than non-commercially or non-specified funding source studies (p=0.024 by χ2 test). We examined changes in reporting of funding, and found that disclosure of funding source increased significantly over the period 1950–95 (p=0.0027 by χ2 test) from 13% during the '70s, to 16.6% during the '80s, and 28.8% during the '90s.
Our discussion covers the three main issues to arise from the review. Firstly, that the research agenda does appear to mirror the wishes of research consumers. Secondly, some areas remain substantially underresearched and urgently need to be considered. Thirdly, overviews of entire subject areas need to be promoted.
The results of the review, unsurprisingly, show that research into OA of the knee joint has been dominated by pharmaceutical and surgical studies. However, the time series results (table 1) show that research output is changing over time; rising for new innovations (such as education) while stabilising or declining for more established interventions (such as NSAIDs). Furthermore, results show that these shifts in the research “agenda” for OA of the knee joint are in the same direction as calls for change by consumers (Tallon D,et al, unpublished data).10This is encouraging as it shows that the research agenda does mirror consumer wishes (even if not directly influenced by them), despite commercial, researcher, and publication biases. This interpretation may be an artefact rather than a true change: a result of literature databases having wider and more comprehensive coverage of publications for more recent years. But the steady growth in research output into non-pharmaceutical and non-surgical interventions (both in absolute and relative terms), suggests this is not the case.
We realise that this review would have benefited from a second independent observer for study selection and data extraction. It might also be argued that applying a quality assessment measure (for example, Chalmers, Jadad scales)11 12 would have been useful, but a standard assessment for different study types has not been available until recently.13 Also, research suggests that the use of quality assessment instruments may be limited.14 15 We were only able to examine English language articles, and studies have shown that this may introduce bias into reviews.16Furthermore, we appreciate that the excessive use of literature databases may be problematic. The increase in research output we have seen may in part be due to databases having incomplete temporal coverage (for example, they do not include all back issues of journals). Also, our search strategy may not have been sensitive enough to identify all relevant articles. So we cannot claim the research base that we have identified represents the entire literature base. However, we made our search strategy deliberately open, avoiding specific intervention terms, as we wanted to avoid biasing results towards any one intervention or relying too much on the database tools to select relevant research.
It is already known that the research base is influenced by a number of factors, and such biases have been shown by previous research.8 9 So we have not concentrated our discussion on these areas. But our results confirm previous findings that factors such as commercial funding influence the published research base.
Having highlighted some encouraging trends in the research agenda, we can nevertheless make some clear recommendations for further research based upon our results. We have identified three main areas that appear to be underresearched; certainly underrepresented in our search results. Firstly, few studies have been undertaken comparing different interventions—for example, conservative treatment compared with surgery. A few examples do exist,17-20 but this area warrants further attention. Secondly, OA of the knee joint, like many other chronic conditions, requires long term management. It is unlikely that any one treatment will be effective at all times. It is therefore necessary to know more about the progression of the disease and which interventions are most effective at different stages for individual patients. Thirdly, it is evident that for the goal of more comparative evidence to be achieved, more evidence from randomised controlled trials (RCTs) needs to be collected for all interventions, with the exception of drugs where the RCT method is already standard practice. Alternatively, reviewers may have to start widening their inclusion criteria to allow high quality observational data to be included. If reviews, and Cochrane reviews in particular, are to reflect the reality of patient treatment and preferences then this problem must be considered.
As we have shown, the research agenda does appear to mirror consumer wishes (at least for literature on interventions for OA of the knee joint). However, several areas still require further evaluation. The areas we have flagged as underresearched may be difficult to research, which might explain why they have yet to be investigated. However, these areas are not impossible to research. For example, the continuing existence of surgical waiting lists in the United Kingdom means that surgery can be compared with other interventions, with people being randomly allocated to alternative treatments while on the waiting list for surgery.
Our final point relates to methodology. We feel that the type of overview of the literature conducted here, coupled with consumer surveys, will become increasingly important for the identification of areas of underresearch. As medical science advances and fragments it will be necessary to retain some form of overview of the research base to ensure that evidence based medicine is representative of consumer needs.
We would like to thank the OARSI group for all the information they provided. We would also like to thank Matthias Egger for his invaluable help with planning this project.
Bristol is the lead centre of the MRC—HSRC.
Funding: This study was funded from core funds of the Medical Research Council—Health Services Research Collaboration.