An Open-Source, API-Based Framework for Assessing the Correctness of Code in CS50

Abstract

We present check50, an open-source, extensible tool for assessing the correctness of students’ code that provides a simple, functional framework for writing checks as well as an easy-to-use API that abstracts away common tasks, among them compiling and running programs, providing their inputs, and checking their outputs. As a result, check50 has allowed us to provide students with immediate feedback on their progress as they complete an assignment while also facilitating automatic and consistent grading, allowing teaching staff to spend more time giving tailored, qualitative feedback.

We have found, though, that since introducing check50 in 2012 in CS50 at Harvard, students have begun to perceive the course’s programming assignments as more time-consuming and difficult than in years past. We speculate that the feedback that check50 provides prior to students’ submission of each assignment has compelled students to spend more time debugging than they had in the past. At the same time, students’ correctness scores are now higher than ever.