Combining Clickstream Analyses and Graph-Modeled Data Clustering for Identifying Common Response Processes

Esther Ulitzsch, Qiwei He, Vincent Ulitzsch, Hendrik Molter, André Nichterlein, Rolf Niedermeier, Steffi Pohl

Research output: Contribution to journalArticlepeer-review

15 Scopus citations

Abstract

Complex interactive test items are becoming more widely used in assessments. Being computer-administered, assessments using interactive items allow logging time-stamped action sequences. These sequences pose a rich source of information that may facilitate investigating how examinees approach an item and arrive at their given response. There is a rich body of research leveraging action sequence data for investigating examinees’ behavior. However, the associated timing data have been considered mainly on the item-level, if at all. Considering timing data on the action-level in addition to action sequences, however, has vast potential to support a more fine-grained assessment of examinees’ behavior. We provide an approach that jointly considers action sequences and action-level times for identifying common response processes. In doing so, we integrate tools from clickstream analyses and graph-modeled data clustering with psychometrics. In our approach, we (a) provide similarity measures that are based on both actions and the associated action-level timing data and (b) subsequently employ cluster edge deletion for identifying homogeneous, interpretable, well-separated groups of action patterns, each describing a common response process. Guidelines on how to apply the approach are provided. The approach and its utility are illustrated on a complex problem-solving item from PIAAC 2012.

Original languageEnglish
Pages (from-to)190-214
Number of pages25
JournalPsychometrika
Volume86
Issue number1
DOIs
StatePublished - 1 Mar 2021
Externally publishedYes

Keywords

  • action sequences
  • cluster editing
  • complex problem solving
  • response times

ASJC Scopus subject areas

  • Psychology (all)
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Combining Clickstream Analyses and Graph-Modeled Data Clustering for Identifying Common Response Processes'. Together they form a unique fingerprint.

Cite this