Unified view of prediction and repetition structure in audio signals with application to interest point detection

Shlomo Dubnov

Research output: Contribution to journalArticlepeer-review

20 Scopus citations

Abstract

In this paper, we present a new method for analysis of musical structure that captures local prediction and global repetition properties of audio signals in one information processing framework. The method is motivated by a recent work in music perception where machine features were shown to correspond to human judgments of familiarity and emotional force when listening to music. Using a notion of information rate in a model-based framework, we develop a measure of mutual information between past and present in a time signal and show that it consist of two factors - prediction property related to data statistics within an individual block of signal features, and repetition property based on differences in model likelihood across blocks. The first factor, when applied to spectral representation of audio signals, is known as spectral anticipation, and the second factor is known as recurrence analysis. We present algorithms for estimation of these measures and create a visualization that displays their temporal structure in musical recordings. Considering these features as a measure of the amount of information processing that a listening system performs on a signal, information rate is used to detect interest points in music. Several musical works with different performances are analyzed in this paper, and their structure and interest points are displayed and discussed. Extensions of this approach towards a general framework of characterizing machine listening experience are suggested.

Original languageEnglish
Article number4432637
Pages (from-to)327-337
Number of pages11
JournalIEEE Transactions on Audio, Speech and Language Processing
Volume16
Issue number2
DOIs
StatePublished - 1 Feb 2008
Externally publishedYes

Keywords

  • Information rate
  • Interest points
  • Musical structure
  • Recurrence matrix
  • Spectral anticipation
  • Spectral clustering
  • Visualization

ASJC Scopus subject areas

  • Acoustics and Ultrasonics
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Unified view of prediction and repetition structure in audio signals with application to interest point detection'. Together they form a unique fingerprint.

Cite this