Person: Macwilliam, Thomas
Loading...
Email Address
AA Acceptance Date
Birth Date
Research Projects
Organizational Units
Job Title
Last Name
Macwilliam
First Name
Thomas
Name
Macwilliam, Thomas
3 results
Search Results
Now showing 1 - 3 of 3
Publication Scaling Office Hours: Managing Live Q&A in Large Courses(2012) Macwilliam, Thomas; Malan, DavidComputer Science 50 (CS50) is Harvard University’s introductory course for majors and nonmajors alike. So that students have an adequate support structure with which to tackle the course’s weekly programming assignments, we offer weekly “office hours,” during which students can receive one-on-one help from teaching assistants. In Fall 2010 and years prior, office hours were held in a basement-level computer lab. However, this environment did not appeal to staff or students. Moreover, this format for office hours suffered from logistical inefficiencies, repetition of questions among students, and lack of communication among staff, which led to high wait times for students. We relocated office hours in Fall 2011 to dining halls to create a more social and collaborative workspace, with more staff on duty at once. We also developed CS50 Queue, a web- and iPadbased system for managing office hours’ logistics. Overall, the new format proved a success. Attendance at office hours grew more than linearly, with an average of 120 students attending per night, up from 30 students in 2010 despite only a 23% increase in enrollment. Even though Queue enabled us to scale, new logistical challenges arose, and wait times for students still sometimes exceeded an hour. We intend to address those challenges in Fall 2012 in order to reduce wait times to 15 minutes at most.Publication Engaging Students through Video: Integrating Assessment and Instrumentation(Association for Computing Machinery, 2013) Macwilliam, Thomas; Aquino, R.J.; Malan, DavidCS50 is Harvard’s introductory course for majors and non-majors alike. For years, we have posted videos of the course’s lectures and sections online for the sake of review and distance education alike. But students’ experience with these videos has been historically passive. Students have been able to watch the course’s content on demand, rewinding and fast-forwarding at will, but they have not had means to engage interactively with the content or to check their understanding of material while watching videos. Furthermore, while we collected basic usage data (e.g., how many times a video was viewed), we lacked detailed analytics describing, for example, which portions of a video were commonly skipped or watched multiple times by students. To make videos more immersive and engaging for students, we developed CS50 Video, an open-source video player for desktop and mobile devices. CS50 Video allows instructors to integrate assessment questions to be answered by students at their own pace or at specific points in time directly into a video player. CS50 Video also allows students to search over video transcripts to find content easily as well as view videos at variable playback speeds (in order to make videos more accessible for ESL learners). Finally, CS50 Video integrates with third-party analytics solutions to allow instructors to view detailed usage statistics describing how students are interacting with videos (e.g., which videos or portions of videos are commonly watched or skipped over). We have deployed CS50 Video to students taking CS50 online and have obtained preliminary results. Because CS50 Video stores responses to questions server-side, we have been able to track students’ performance on in-video assessments. Thus far, we have observed that only 28% of students who watch online videos have engaged with assessment questions. Students who answer an assessment question incorrectly on their first attempt will often try again until reaching a correct answer, with 84.5% of correct answers reached in at most three attempts. We next plan to analyze the effects of in-video assessments on students’ mastery of material and introduce A/B-testing functionality for questions. We also plan to use students’ performance on assessments to understand the topics with which students struggle.Publication Streamlining Grading toward Better Feedback(Association for Computing Machinery, 2013) Macwilliam, Thomas; Malan, DavidCS50 is Harvard University's introductory course aimed at majors and non-majors alike. Each week, students complete programming assignments and have traditionally received feedback from staff in the form of comments on PDFs of their code. Staff have historically reported spending significant amounts of time grading because of bottlenecks that included generating PDF documents and manually emailing feedback to students. Because we preferred that staff spend less of their time on grading logistics and more time providing feedback and helping students online or in person, we set out to improve the efficiency of the grading process. In Fall 2012, we developed and deployed CS50 Submit, a web-based utility through which staff can leave feedback for students via inline "sticky notes." Following the introduction of CS50 Submit, staff reported grading for 10% fewer hours (i.e., 42 minutes) per week and 13% fewer minutes (i.e., 4 minutes) per student, even while providing as much or more feedback. Meanwhile, we observed significantly higher levels of engagement with the course's online discussion board among staff, suggesting a more favorable distribution of staff workload. With CS50 Submit, we have also been able to audit exactly how much time staff spent grading each week in order to identify additional bottlenecks. Using CS50 Submit, we also observed that, on average, 9% of students each week never read their graders' comments, with a peak one week of 14%. The number of students who did not read feedback increased with time, which has led us to question whether asynchronous, textual comments are the most effective feedback mechanisms for students. In future terms, we plan to experiment with in-person, interactive means of delivering feedback to students. In this paper, we present CS50 Submit and the insights it has yielded into the behavior of students and staff alike.