Intermodality in Multimodal Learning Analytics for Cognitive Theory Development: A Case from Embodied Design for Mathematics Learning

Multimodal Learning Analytics (MMLA) grant us insight into learners’ physiological, cognitive, and behavioral activity as it unfolds. In this chapter, we query the relations among modalities, intermodality, in the context of a designbased research program studying the relations between learning to move in new ways and learning to think in new ways. In the first part, we reflect on how different methods have afforded purchase on the investigation, development, and elaboration of theoretical claims about the multimodal enactment of cognitive events, culminating in the use of Recurrence Quantification Analysis (RQA) to quantify the microgenesis of stable new patterns in hand movement and gaze. In the second part, we analyze an RQA case study spanning across hand and gaze modalities to examine the emergence of intermodal coordination at a critical moment in the mathematical task. We conclude with implications and open questions around intermodality in embodied learning.