Objective: To develop new methods of evaluating face validity in the context of a revised final professional examination for medical undergraduates, organized on three sites, over 2 days.
Methods: The opinion of the students and examiners was surveyed by Likert-style questionnaires, with additional open comments. Expert opinion was gathered from external examiner reports and a recent Quality Assurance Agency (QAA) Subject Review Report.
Results: The questionnaires had an overall response rate of 84%. Internal reliability, assessed by comparing responses to appropriate questions, was good with an equivalence of 45% (weighted kappa 0.54) for the students and 33% (weighted kappa 0.41) for the assessors. There was little evidence of inconsistency between days or sites. The majority of the opinions from the students, examiners and external experts were positive. Negative comments related to time pressure and case mix.
Conclusion: The measurement of face validity proved feasible and valuable and will assist in the further development of the course and the examination.