Inter-rater reliability (IRR) has long been used in many different scientific fields to measure the reliability of the classification of subjects. In simulation, checklists and rubrics are common tools used to assess a learner’s performance, and the use of inter-rater agreements are used to check and maintain assessment integrity across large class sizes with multiple evaluators.

Join Dr. Backlund as he presents the concept of IRR along with the data considerations needed to construct a proper IRR plan and cases when different agreement coefficients should be used.