On modeling affect in audio with non-linear symbolic dynamics

Pauline Mouawad, Shlomo Dubnov

Research output: Contribution to journalArticlepeer-review

2 Scopus citations


The discovery of semantic information from complex signals is a task concerned with connecting humans' perceptions and/or intentions with the signals content. In the case of audio signals, complex perceptions are appraised in a listener's mind, that trigger affective responses that may be relevant for well-being and survival. In this paper we are interested in the broader question of relations between uncertainty in data as measured using various information criteria and emotions, and we propose a novel method that combines nonlinear dynamics analysis with a method of adaptive time series symbolization that finds the meaningful audio structure in terms of symbolized recurrence properties. In a first phase we obtain symbolic recurrence quantification measures from symbolic recurrence plots, without the need to reconstruct the phase space with embedding. Then we estimate symbolic dynamical invariants from symbolized time series, after embedding. The invariants are: correlation dimension, correlation entropy and Lyapunov exponent. Through their application for the logistic map, we show that our measures are in agreement with known methods from literature. We further show that one symbolic recurrence measure, namely the symbolic Shannon entropy, correlates positively with the positive Lyapunov exponents. Finally we evaluate the performance of our measures in emotion recognition through the implementation of classification tasks for different types of audio signals, and show that in some cases, they perform better than state-of-the-art methods that rely on low-level acoustic features.

Original languageEnglish
Pages (from-to)1727-1740
Number of pages14
JournalAdvances in Science, Technology and Engineering Systems
Issue number3
StatePublished - 1 Jan 2017
Externally publishedYes


  • Dynamical invariants
  • Embedding
  • Information rate
  • Nonlinear dynamics
  • Recurrence plots
  • Symbolization
  • Variable Markov Oracle

ASJC Scopus subject areas

  • Engineering (miscellaneous)
  • Physics and Astronomy (miscellaneous)
  • Management of Technology and Innovation


Dive into the research topics of 'On modeling affect in audio with non-linear symbolic dynamics'. Together they form a unique fingerprint.

Cite this