July 2012: OMP and the RCT

Teaching the One Minute Preceptor: A Randomized Controlled Trial, Furney SL, Orsini AN, Orsetti KE, Stern DT, Gruppen LD, Irby DM. J Gen Intern Med. 2001; 16: 620-624. 

In this article, Furney et al. use a randomized, controlled trial (RCT) to examine the link between a monthly educational intervention – a 1-hour workshop on the One-Minute Preceptor (OMP) – and residents’ teaching skills. To measure the effectiveness of this intervention, they captured pre-post perspectives from learners on residents’ teaching and residents’ self-reported use of teaching skills.

The authors conclude that their intervention improves residents’ abilities to provide feedback and motivates learners to increase their outside reading. In the limitations section, authors note that the improvement in outcomes may not be due to the content of the one-hour educational intervention but instead related to the fact that any session on teaching skills was offered. In a sense, a structured session in which residents were reminded that teaching was an important part of their jobs was enough to mitigate “teacher fatigue”, or deterioration of teaching skills that can occur without presence of an intervention. 

While the RCT is largely a gold-standard for identifying causality in research, the limitations section of each educational research RCT paper is a reminder of the complexities of identifying and defining the specific agents in education. In evaluating the nature of evidence in both clinical and educational research, Patricio and vaz Carneiro¹ point out that there exist complexities in educational interventions, studies, and circumstances that can be difficult to capture and even more difficult to interpret. They write that “the evidence on the success or failure of the intervention may be less clear in medical education studies because the establishment of causal relationship between the intervention and outcomes may be difficult (sometimes impossible)” (p. 480). In other words, defining and measuring various aspects of education can be a formidable and, sometimes, impossible exercise.

In the current article, Furney et al. provide evidence that a monthly intervention demonstrating the OMP improved residents’ teaching skills, from the perspectives of the learners and the teachers. Is the study conclusive? No. But it does provide a solid step in the body of evidence investigating the development of our resident teachers. It is this, and not the objective RCT, that should define a successful venture in quantitative educational research.

Bottom Line:

RCTs are possible and important in medical education research. However, even an RCT cannot guarantee a causal link given the ambiguity of education. It should be the goal of educational researchers to accurately define their study’s components and limitations so that others may continue the exploration. 

Reference:

1. Patricio M, Vaz Carneiro A. Systematic reviews of evidence in medical education and clinical medicine: Is the nature of evidence similar? Med Teach. 2012; 34: 474-482.