The Dawn of AI in Medical Education

In this webinar, we discussed the use of large language models (LLMs) to automate medical student OSCE note grading.

Learning Objectives:

  • Describe the time and cost implications of traditional OSCE note grading and how LLMs can streamline the process.
  • Compare the grading accuracy of an LLM-based system with traditional human grading methods by analyzing performance data from real-world OSCE note evaluations.
  • Outline the key components of an LLM-powered OSCE grading pipeline, including student note input, rubric integration, and automated scoring output, and discuss potential limitations and areas for improvement.
  • Identify opportunities to integrate LLMs into their own educational and clinical workflows by examining the OSCE grading project as a model for innovation in emergency medicine education.

 

Authors

  • Christopher J. Nash, MD, MEd

    Assistant Professor, Emergency Medicine

    Duke University Hospital

    Christopher J. Nash, MD, MEd, is an assistant professor and core faculty in the department of emergency medicine (EM) at Duke University. He completed a fellowship in medical education at Mass General and pursued additional training through the SAEM ARMED MedEd program. He previously served as a Fellow Editor-in-Training for AEM Education & Training and is currently a fellow in the NBME SEEF program. His work focuses on the intersection of technology, education, and innovation, leveraging AI, simulation, and emerging digital tools to enhance clinical training and assessment. He leads several innovative research initiatives that bridge academic theory with frontline practice.

  • Tama Thé, MD

    Assistant Professor, Pediatric Emergency Medicine

    University of Kentucky

    Tama Thé, MD, is an assistant professor of pediatric emergency medicine at the University of Kentucky. He is fellowship-trained in point-of-care ultrasound and serves as the pediatric emergency ultrasound director. He is a third-year clinical medicine course director, course director for the fourth-year emergency ultrasound elective, and host of The MDM, a podcast about the ways medicine is adapting to the modern world.

  • Candace Pau, MD

    Family Physician

    Kaiser Permanente Bernard J. Tyson School of Medicine

    Candace Pau, MD, is a family physician and the founding Faculty Director of Simulation at the Kaiser Permanente Bernard J. Tyson School of Medicine, where she is responsible for the development and implementation of a robust, innovative simulation-based curriculum, encompassing simulation activities for both instruction and assessment at the undergraduate medical education level. Her research interests include competency-based medical education, simulation-based assessment, clinical reasoning, and the transition to residency. She has extensive direct teaching and curricular design experience, including the use of flipped classrooms, experiential learning, and serious games. Dr. Pau received her medical degree from Stanford University School of Medicine and completed her residency training at the Kaiser Permanente Napa-Solano Family Medicine Residency Program.

  • Nayef Chahin, MD

    Associate Program Director, Intern Recruitment

    Children's Hospital of Richmond at VCU

    Nayef Chahin, MD, is a member of the American Academy of Pediatrics and a fellow of the NBME Strategic Educators Enhancement Fellowship (SEEF) second cohort. He is also an Academic Pediatric Association Research Scholar from cohort six. Dr. Chahin serves as associate program director for intern recruitment in the Pediatric Residency Program and the Neonatal Perinatal Fellowship. He has served as the Richmond delegate for the Virginia AAP Chapter and currently co-chairs the APA Region IV Planning Committee and is a member of the ONTPD Planning Committee. Clinically, Dr. Chahin is an attending faculty member in the level IV neonatal intensive care unit, the Neonatal Continuing Care Program, the Regional neonatal Follow-up Program, and the CHoR Air and Ground NICU Transport Team. He leads the CHoR NICU Unplanned Extubation Committee.

Related Media