A Koopman Approach to Understanding Sequence Neural Models

Ilan Naiman, Omri Azencot

Research output: Working paper/PreprintPreprint

29 Downloads (Pure)

Abstract

We introduce a new approach to understanding trained sequence neural models: the Koopman Analysis of Neural Networks (KANN) method. Motivated by the relation between time-series models and self-maps, we compute approximate Koopman operators that encode well the latent dynamics. Unlike other existing methods whose applicability is limited, our framework is global, and it has only weak constraints over the inputs. Moreover, the Koopman operator is linear, and it is related to a rich mathematical theory. Thus, we can use tools and insights from linear analysis and Koopman Theory in our study. For instance, we show that the operator eigendecomposition is instrumental in exploring the dominant features of the network. Our results extend across tasks and architectures as we demonstrate for the copy problem, and ECG classification and sentiment analysis tasks.
Original languageEnglish GB
PublisherarXiv:2102.07824 [cs.LG]
StatePublished - 2021

Keywords

  • cs.LG
  • math.DS

Fingerprint

Dive into the research topics of 'A Koopman Approach to Understanding Sequence Neural Models'. Together they form a unique fingerprint.

Cite this