Modeling Engagement in Self-Directed Learning Systems Using Principal Component Analysis

This paper studies students engagement in e-learning environments in which students work independently and solve problems without external supervision. We propose a new method to infer engagement patterns of users in such self-directed environments. We view engagement as a continuous process in time, measured along chosen axes that are derived from student data in the system using unsupervised learning (Principal Component Analysis). We construct a trajectory of user activity by projecting the user's scores along the selected PCs at regular time intervals. This approach is applied to a popular e-learning software for K12 math education that is used by thousands of students worldwide. We identify cohorts of users according to the way their trajectory changes over time (e.g., monotone up, monotone down, and constant). Each of the cohorts exhibits distinct behavioral dynamics and differed substantially in the amount of time users spent in the e-learning system. Specifically, one cohort included students that dropped out of the system after choosing very difficult problems that they were not able to complete, while another cohort included students users that chose more diverse problems and stayed longer in the system. In future work, these results can be used by teachers or intelligent tutors to track students' engagement in the system and decide whether and how to intervene.