OBJECTIVE To identify factors that influence medical students’ perceptions of the quality of a clinical skills course; to apply these factors to the course at one hospital; to measure the effect of this change.
DESIGN Cross sectional questionnaire survey; application of identified factors; repeat questionnaire survey.
SETTING Three teaching hospitals and five district general hospitals in north east England.
SUBJECTS Third year medical students attending locomotor clinical skills courses in two consecutive years.
MAIN OUTCOME MEASURES Score awarded by students in five categories; numbers of patients seen by each student; comparisons with other clinical skills weeks.
RESULTS Response rates were 71 of 150 and 89 of 161. Factors associated with a high awarded score were: organisation of the course by a rheumatologist (p<0.01); teaching from a rheumatologist (p<0.01); higher number of patients seen (r=0.76). Mean number of patients seen varied widely, from 7 per student at one hospital to 20.4 at another. Teaching hospitals scored poorly. In the second year, after making changes at one teaching hospital the mean total score improved (p<0.01), and students saw more patients (p<0.01). The ranking of this hospital rose from 6 to 1. The additional cost of the modified course was £640 per student.
CONCLUSIONS The standard of teaching of locomotor clinical skills varies widely and can be improved by application of factors identified in this survey, although additional costs are incurred.
- undergraduate teaching
- musculoskeletal system
- clinical skills
Statistics from Altmetric.com
Musculoskeletal teaching varies considerably between different medical schools. Newcastle University Medical School started a new undergraduate curriculum in the academic year 1994/5. All medical students are taught basic clinical skills at the beginning of their third year. The course is divided into eight weeks, with a different system of the body taken as the theme for each week. Students are taught musculoskeletal clinical skills in week 7, at one of eight different hospital sites around the old Northern Region: five district general hospitals and three teaching hospitals. In some cases, this is the first formal rheumatology teaching to take place at these hospitals, and new money has been allocated.
Our impression from contact with the students later in the year was that students’ knowledge and confidence in locomotor history taking and examination varied, and that this might relate to different experiences in the clinical skills week. We were also concerned about the structure and content of the course at one hospital, for which we provided teaching but had no other input.
The aims of this study were: (1) To identify the factors that influence medical students’ perceptions of the quality of musculoskeletal clinical skills teaching. (2) To apply these factors to the musculoskeletal clinical skills course at one hospital. (3) To test the effect of these changes.
PHASE ONE: 1996
A survey was undertaken to identify differences in teaching in different hospitals and components of the teaching that contributed to students’ perceptions of the quality of the course. Students attending the 1996 clinical skills week were surveyed using a structured questionnaire given out and handed in at the next part of the rheumatology course, five months later. Details were requested of the specialty of tutors and the number of patients in whom the student had personally: (a) taken a locomotor history; (b) examined the whole locomotor system; (c) examined part(s) of the locomotor system.
Students awarded marks for each of five elements of the course (total possible score 25) with 1 the minimum and 5 the maximum possible mark. These were: relevance of course content; organisation of the week; standard of teaching; their perception of the ability of tutors to assess individual student performance; approachability of the tutors if students were having difficulties.
The students were also asked to compare the musculoskeletal week with other clinical skills weeks taught at the same hospital and were invited to add comments of their own.
PHASE TWO: 1997
The 1997 clinical skills course at hospital A was changed in format and structure according to factors identified in Phase One. No interventions were made at other sites (although because of alteration in clinical services one teaching hospital (B) did not participate in the 1997 clinical skills course and another district general hospital (I) took its place). The same evaluation questionnaire was then applied at the next rheumatology teaching session, and results were compared with Phase One. Additional costs incurred by the implementation of these changes were calculated.
Calculations were performed using EPI-INFO version 6, using the χ2 and Kruskal-Wallis tests and simple linear regression with significance set at the 5% level.
PHASE ONE: 1996
One hundred and ten of 150 students were present at the next part of the course when the questionnaires were distributed. Seventy one questionnaires were returned, giving a response rate of 65% of those attending or 47% overall.
Table 1 shows that the clinical specialty of course leaders and of teachers varied depending upon the hospital site.
Students reported a wide variation in numbers of patients seen during the week (table 2). Patient contact varied significantly according to the hospital at which the student was taught (χ2, p=0.04). Five students said that they had not taken a history from or examined any patients at all. Four of these were taught at hospital B and one at hospital A. Fifteen students had not taken a locomotor history, 26 had not performed a complete locomotor examination, and five had not performed a partial examination. Only at hospital F did all students report performing at least one locomotor history, one full locomotor examination, and a partial examination.
Students’ rating of the musculoskeletal clinical skills week
Table 2 shows the students ratings of the courses at the different hospitals. The total marks varied widely with a threefold difference in scores. One hospital received the maximum mark from five of six students who returned questionnaires.
Teaching from a rheumatologist was significantly associated with a higher total score awarded (χ2, p<0.01). None of the students awarding the minimum mark for teaching had been taught by a rheumatologist. The total mark correlated with the number of patients seen (r=0.76) and with the perceived standard of teaching (r=0.93).
Organisation of the course by a rheumatologist was also associated with a higher total score awarded (Kruskal-Wallis H=39.07, p<0.01). Teaching was organised by rheumatologists, sometimes in conjunction with orthopaedic surgeons, in each of the four hospitals receiving the highest overall marks, by physicians in those ranked 5–7, and by orthopaedic surgeons alone in the hospital ranked eighth. The teaching hospitals were ranked as the lowest three.
Comparison with other clinical skills weeks
Students views of the locomotor week in comparison with other weeks varied widely. All six students at hospital F rated this week much better than average. At hospitals C, E, and G all students rated this week as average or better, 11 of 12 at hospital H, and 7 of 8 at hospital D thought it was average or worse, and all eight students at hospital B found it slightly or much worse than average. A wide spread of ratings was given by students who attended hospital A.
Free text comments
Thirty students added comments of their own. Ten of these were negative, all from hospitals B and H, and the other 20, from students attending hospitals A, C, F, and G, were positive. Many comments concerned the organisation of the course and perceived attitudes of tutors to the locomotor teaching (table3).
In summary, the students’ opinions of teaching at the eight sites varied widely. Factors associated with a good opinion were: organisation of the course by a rheumatologist; teaching from a rheumatologist; higher number of patients seen.
PHASE TWO: 1997
The students’ perception of the locomotor clinical skills teaching at hospital A was poor in 1996. The mean total score was 16.33/25, and it was ranked sixth of the eight hospitals. For the 1997 course the rheumatology department was given control of the whole week and instituted a number of changes, based on the findings from phase I. The course was organised and delivered by rheumatologists and general practitioners. A structured timetable was provided, specifying the expected content of each session. Students were taught in groups of three in the hospital and six in general practice, with continuity of tutor where practicable. Patients were present for every teaching session and patient contact was maximised. At the end of the week students presented case histories to consultant tutors, and an OSCE was undertaken, with a prize awarded to the winning team.
The same evaluation questionnaire was distributed to the 120 of 161 students attending the next rheumatology teaching day. Eighty nine questionnaires were returned, giving a response rate of 74% of those present and 55.28% overall. The following differences were shown between 1996 and 1997 at hospital A.
Students recalled seeing more patients in 1997 than 1996 (Kruskal-Wallis, p<0.01) (table 4). More histories were taken by each student (p=0.017) and more partial examinations were performed (p<0.01). The number of complete examinations performed by each student was numerically greater but not significantly changed (p=0.095).
Scores and ranking
The mean total score improved from 16.3 to 24.1 and the ranking improved from 6 to 1 (fig 1). All students rated the locomotor week as slightly or much better than the other weeks of the course at the same hospital.
Provision of the 1997 course was considerably more expensive than the previous year as shown in table 5. The major additional costs incurred were medical staff time and new patient appointments cancelled. Using the midpoints of the relevant scales, we calculated salary costs per hour as follows: consultants £24.09; specialist registrars £12.56; senior house officers £10.50.
Consultants provided 66 hours of teaching time and eight hours of preparation, costing £1783. Eighty one hours of specialist registrar time were used—57 hours of teaching and 24 hours of preparation time—at a total cost of £1017. A senior house officer taught for six hours at a cost of £63. In 1996, 24 hours of consultant teaching and 12 hours of registrar teaching time were used, with a total cost of £729. Costs to the medical division for 1996 were not calculated. For 60 new patient appointments at £299 each, the cost for cancellation of a week’s clinics in 1997 was £17 940; no clinics were cancelled in 1996. Three private patient clinics were also cancelled: the costs of this to individual consultants and to the hospital have not been calculated. Establishment costs, for example heating, lighting, building and nursing costs were unchanged from the previous year and have not been calculated. Additional costs in Phase II were therefore incurred by an extra 125 hours of staff time, costing £2134, and lost revenue from new patient appointments at a cost of £17 940. For 32 students, this is an additional cost of £640 per student (equivalent to 27 hours of consultant time) for the 1997 course in comparison with 1996.
Locomotor complaints are common in all fields of medicine.1-3 Sound techniques in history taking and examination are necessary to make reliable diagnoses in this system perhaps even more than others. If students do not acquire skills and confidence in this area then it is probable that their future practice will be adversely affected, and their patients with locomotor complaints will receive suboptimal treatment.
The standard of the musculoskeletal clinical skills week has been shown in Phase One to vary widely throughout the region, both in terms of student perceptions and according to amount of patient contact. Some students had an excellent introduction to the locomotor system and feel they have had the opportunity to acquire the necessary skills. Important factors contributing to a high overall mark awarded by students include the number of patients seen, and organisation and delivery of the teaching by rheumatologists. The three teaching hospitals in our region performed less well than the five district general hospitals. Two of the teaching hospitals did not have any rheumatological input to their teaching, which was also the case in the lowest ranked district general hospital. Musculoskeletal teaching is probably a special case in that compared with other body systems it has not been well taught or considered of importance in the past.4-6 Consequently other physicians have not given the same priority to locomotor skills as they have to other systems.1 7 Until musculoskeletal teaching achieves the same status, it is probably best taught under the direction of rheumatologists.
The fact that five students cannot recall taking a locomotor history or performing a locomotor examination on any patients is worrying. It is generally agreed that in rheumatology, as in many other specialties, accuracy of diagnosis is highly dependent on the patient’s history. Fifteen students did not recall taking a history from any patient with locomotor disease during this course: it is probable that their subsequent history taking will be deficient.
Students’ views and recall are used in this study to evaluate the clinical skills courses at the eight different hospital sites. An objective assessment of the standard of each student’s musculoskeletal clinical skills performance at the end of the course would have provided additional useful information but was not possible within the medical school’s existing examination schedule.
This study does not show how many students missed opportunities for patient contact by failure to attend teaching sessions. Attendance may, of course, be influenced by student expectations of the content and value of the teaching.
The application of the factors identified in Phase One led to an improvement in the course at hospital A between 1996 and 1997, both in terms of student perception and in numbers of patients seen by each student. The 1997 course was also more enjoyable to teach, although it required considerable commitment from individual tutors and was more expensive to provide.
The high standard set by some hospitals in this region is encouraging, and deficiencies in other areas need to be tackled. We have shown that it is possible to improve the delivery of clinical skills teaching by attention to organisational factors, although high additional costs are incurred.
Conflicts of interest: both authors are rheumatologists and tutors on the clinical skills course at the University of Newcastle upon Tyne.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.