Does continuing medical education in general practice make a difference?

Methods

We searched the bibliographic databases of Medline, BIDS, ERIC, and Embase between 1990 and March 1999 for (a) systematic reviews of continuing medical education, (b) systematic reviews of postgraduate continuing medical education for general practitioners, and (c) postgraduate educational interventions based on general practice. (The term “postgraduate” is taken to mean educational events occurring after completion of general practice vocational training.) We included intervention studies if they contained a robust evaluation, which examined either the effects of the educational event on subsequent doctor behaviour or patient outcomes. We then retrieved selected references from these papers. The papers were graded by applying a standard hierarchy of evidence, with randomised controlled trials at the top and descriptive studies at the bottom.

Results

The expectations of continuing medical education

The primary purpose of continuing medical education is to maintain and improve clinical performance.9 Recertification and reaccreditation are part of an international trend to shift the purpose of continuing medical education towards assuring adequate performance.10 The world in which doctors work has changed enormously. Increasing consumerism and patient empowerment, growing accountability to external bodies, and more emphasis on efficiency and effectiveness have led to an intolerance of variance in medical practice. Quality assurance and the maintenance of standards have become powerful forces for change.11 In an evidence based medical world it would seem prudent therefore for those planning general practitioners' education to choose educational methodologies that have been shown to work, and to evaluate those that have not.

Systematic reviews of continuing medical education

There is a growing international consensus on what forms of continuing medical education are most effective in stimulating behaviour change. Systematic reviews12^–15 of the educational literature found that although there were comparatively few rigorous evaluations of educational interventions, there were sufficient studies showing that continuing medical education could improve clinical performance and patient outcomes, indicating which methods were best at bringing about change in doctors' behaviour.

The most effective methods derived from these reviews include learning linked to clinical practice, interactive educational meetings, outreach events, and strategies that involve multiple educational interventions (for example, outreach plus reminders). Less effective strategies include audit, feedback, local consensus processes, and the influence of opinion leaders. The least effective methods are also the most commonly used in general practice continuing medical education—namely, lecture format teaching and unsolicited printed material (including clinical guidelines).

Some reviews propose models for ensuring medical behaviour change. 12 13 Three sequential strategies are described. These are:

• Consideration of predisposing factors, which prepare doctors for change

• Identification of enabling factors by which new knowledge and skills are related to the learner's work environment

• Reinforcement of new behaviour through the use of reminders and feedback.

Innovations, guidelines, and behaviour change

Lomas has described many of the factors that contribute to change in doctor behaviour.16 These include educational, personal, patient based, and economic factors. The context in which doctors work may have a profound effect on their willingness and readiness to change. 16 17 Local perceptions of an innovation may affect subsequent behaviour change. Factors such as the relative advantage the innovation offers over existing practice, its complexity, and its trialability are all important considerations.18

Grimshaw and Russell studied the relative effectiveness of different strategies used to implement clinical guidelines.19 They found that the most successful strategies involved local rather than national guideline development and dissemination combined with a focus on prompting (or reminding) the doctor during the consultation. The least effective methods were those most commonly used—namely, national guideline development combined with unsolicited distribution.

Reviews of postgraduate continuing medical education for general practitioners

Reviews of effective educational methodologies in primary care generally concur with the findings of wider literature reviews of continuing medical education. Combinations of educational interventions were found to be better than single interventions. 20 21 Wensing et al found that organisational and management support were important additional factors in changing behaviour.22 Several authors highlighted the importance of relating educational activity to the work that doctors do. 23 24 Peer review and group learning models were proposed as particularly relevant in general practice settings.22

Needs assessment

Prior needs assessment is important for informing and directing the educational process.21 Relying on doctors to identify their own learning needs, however, may be problematical as Tracey et al found in a study of doctors in New Zealand.25 They found a poor correlation between doctors' self assessment of their knowledge and their subsequent performance in objective tests of their knowledge. Given the freedom to select which educational events to attend, doctors often choose not to stray outside their “comfort zone.” A randomised controlled trial of continuing medical education in 1982 showed that if given the opportunity clinicians choose educational events that fit in with what they already know.26 Furthermore, when the same clinicians were encouraged to cover topics that were not their preferred choice, their quality of care rose significantly compared with a control group. Needs assessment should not therefore be based entirely on self assessment but should use evidence from a range of sources.27

Educational intervention studies in general practice

We found 1032 articles describing educational or audit activities in general practice between 1990 and March 1999. Of these, 69 papers described educational interventions that met the evaluative criteria outlined above. These included 18 papers describing audits with educational interventions, and 51 papers detailing educational studies. Twenty eight studies were of a before/after design,16 were randomised controlled trials, and 15 were controlled trials. Of the remainder, six were exit only studies of sufficient robustness to be included, and four were qualitative evaluation studies.

Seventeen of the 18 audits showed a positive influence on doctor behaviour of which only one included data showing the behaviour change was sustained.28 Publication bias is likely to have influenced the rate of reporting of positive findings. Two audit studies described significant event audits. This has been shown to be an effective model for linking educational intervention, practice, and behaviour change.29

Seven audit studies described interventions that involved setting standards by local consensus. This has been shown to be a very appropriate method for implementing guidelines in general practice.19

The 51 educational intervention papers covered a wide variety of learning events based on general practice. The methodologies included 14 studies using multiple educational strategies and 37 using single strategies. Of these, seven studies did not succeed in changing doctor behaviour. Useful lessons can be learned from these studies with negative outcomes. For example, one recent study found no effect from unsolicited feedback on doctors' prescribing behaviour, and concluded that unsolicited and non-personalised feedback was ineffective.30 A similar but effective intervention has been described by Winkens et al, the difference being that the feedback to doctors was preplanned and personalised.31

A randomised controlled trial that tested a patient centred approach to the care of patients with type 2 diabetes failed to produce sustained behaviour change because the educational intervention was too complex.32 The authors recommended the piloting of complex educational interventions before embarking on large studies.

A much quoted study of a multifaceted educational intervention to improve doctors' management of depression and thus reduce suicide rates on the island of Gotland near Sweden showed very positive early results including a reduction in the suicide rate.33 A 3 year follow up study, however, showed that the doctors' management of depression had deteriorated and that the suicide rate had returned to almost preintervention levels.34 The authors stressed the importance of reinforcing learning. Few studies in this series of 69 did any follow up beyond3 months.

A group learning approach was the main educational methodology in seven studies. Moran et al describe an interesting learner based group, which was designed to help poorly performing general practitioners.35 They were placed in a learning group with10 other doctors as controls. The group met for 10 sessions. Follow up included clinical care, preventative care, and the use of drugs at 6 and 18 months. The study subjects were initially scoring much lower than controls but later improved significantly during the continuing medical education programme.

Two studies looked at the use of computers as a decision support aid36 and reminder system.37 Both studies showed that the use of computers during consultations could both initiate and maintain behaviour change. These findings are similar to those of Grimshaw and Russell who studied the factors leading to the dissemination and application of clinical guidelines in practice.19

Why do doctors change their behaviour?

In addition to the literature on doctor behaviour described above, two recent studies shed further light on why general practitioners change their clinical behaviour. Allery et al used analysis of critical incidents to study why doctors and consultants change their clinical behaviour (for example, changes in therapeutic management or use of investigations).38 They found that most changes were brought about by a combination of factors. Formal continuing medical education was partly responsible for behaviour change in only one third of cases. Organisational factors and contact with other healthcare professionals were equally important factors.

Armstrong et al studied why doctors change their prescribing behaviour.39 They postulated three models of behaviour change (box). These models have a face validity but need to be tested more rigorously.

Evaluation of continuing medical education for doctors

The most striking feature of this review is the lack of robust evaluations of general practice based educational interventions. Of those who did produce “generalisable” findings, a very small proportion of the evaluative studies were designed to test whether behavioural change was sustained. Is this something to worry about or does it simply reflect the problems inherent in educational research and evaluation?

Grant funding for educational research is not easy to obtain and evaluation can consume a lot of time and resources.41 T42 The designers of educational programmes may prefer, therefore, to spend their limited funds on developing and implementing educational innovations rather than evaluating them. Educational evaluation studies are not often published in general readership journals. They are often rejected because they are not sufficiently rigorous or are not deemed to be of “general interest.” Controlled trials of educational events are particularly difficult. There are often problems finding appropriate control groups. Furthermore; evaluation studies are not easily generalised to other settings because of the singular nature of each learning environment.

Despite these difficulties, evaluation remains an important part of the educational cycle. Widespread dissemination of educational ideas is problematic without it, and other workers may be reluctant to try innovations that have not been rigorously tested. Valuable lessons from interventions with negative outcomes may be lost. 32 34

The way forward

Educators of doctors should take account of the literature on effectiveness of educational interventions as described above. Guides to the planning and evaluation of educational events are available. 42 43

General practice educational activity should be based on the work that doctors do. Standard (and significant event) audits have been shown to be effective strategies for behaviour change if they include targeted feedback. This review has highlighted the importance of building reinforcement strategies into educational planning. Group and peer review type interventions have also been shown to be feasible and effective.

There are several very positive trends in continuing medical education in primary care, which seem to incorporate both adult learning principles and the findings of the “what is effective in continuing medical education” literature. Calman, for example, has proposed that the present financially driven credit based system in the United Kingdom be replaced by a new approach in which continuing education, audit, research, and clinical effectiveness are aligned in a unified educational strategy.44 The educational programme as envisaged will be self directed, practice based, and multiprofessional. There are similarities between this proposed system and the quality assurance and continuing education programme in Australia.45 Much, however, of the adult learning theory underlying these and other innovations has not been adequately evaluated.2 These ground breaking programmes will be all the more valuable therefore if their coordinating bodies establish rigorous and continuing evaluation.