Adopting Computer Vision for Instructional Support in Science Education with Informed Technological and Pedagogical Considerations

Abstract

Pedagogical shifts towards student-centered learning makes science teaching more challenging for teachers. As a result, they might find themselves unable to target higher pedagogical goals or restricted in their capacity to support students of various proficiency levels. This presents an opportunity to leverage computer vision technology as an instructional support to assist teachers. However, it is currently unclear how computer vision might be deployed in a real-world setting. The detailed list of pedagogical tasks which can be offloaded by teachers is unknown, and the challenges of implementing computer vision ecologically have not been studied extensively. To this end, I conducted three separate investigations. The first investigation takes on a theoretical approach to document teaching noticing in science education and to compare the affordances of computer vision techniques with practical requirements of teacher noticing. The second investigation qualitatively investigates noticing practices of science teachers and unpacks design considerations of a computer vision system that can serve as an instructional support tool. The last investigation empirically studies computer vision models and explores how computer vision can support science teachers in a real-world setting. In terms of technological considerations, results revealed 1) the high specificity of contemporary computer vision techniques and their inability to carry out interpretations of gathered observations, 2) the need to process and aggregate computer vision outputs for teacher consumption, and 3) the challenges of building computer vision models that perform accurately and reliably in a real-world setting. In terms of pedagogical considerations, results found that 1) there exist at least 12 generalized classifications of noticing features that science educators notice during student-centered learning, 2) there are seven major categories and 36 minor categories of student activity that science teachers typically observe during instruction-based practical work, and 3) providing feed-forward feedback is how computer vision can support teachers during instruction-based practical work. Overall, this work contributes to our real-world understanding of how computer vision could be adopted for instructional support in science education, informed by technological and pedagogical considerations.