Effective model representation by information bottleneck principle

Ron M. Hecht; Elad Noor; Gil Dobry; Yaniv Zigel; Aharon Bar-Hillel; Naftali Tishby

doi:10.1109/TASL.2013.2253097

Effective model representation by information bottleneck principle

Ron M. Hecht, Elad Noor, Gil Dobry, Yaniv Zigel, Aharon Bar-Hillel, Naftali Tishby

Department of Biomedical Engineering

Research output: Contribution to journal › Article › peer-review

7 Scopus citations

Abstract

The common approaches to feature extraction in speech processing are generative and parametric although they are highly sensitive to violations of their model assumptions. Here, we advocate the non-parametric Information Bottleneck (IB). IB is an information theoretic approach that extends minimal sufficient statistics. However, unlike minimal sufficient statistics which does not allow any relevant data loss, IB method enables a principled tradeoff between compactness and the amount of target-related information. IB's ability to improve a broad range of recognition tasks is illustrated for model dimension reduction tasks for speaker recognition and model clustering for age-group verification.

Original language	English
Article number	6480793
Pages (from-to)	1755-1759
Number of pages	5
Journal	IEEE Transactions on Audio, Speech and Language Processing
Volume	21
Issue number	8
DOIs	https://doi.org/10.1109/TASL.2013.2253097
State	Published - 22 May 2013

Keywords

Information bottleneck method
information theory
speaker recognition
speech recognition

ASJC Scopus subject areas

Acoustics and Ultrasonics
Electrical and Electronic Engineering

Access to Document

10.1109/TASL.2013.2253097

Cite this

@article{bd56265358234676891b03e2aeb6c277,

title = "Effective model representation by information bottleneck principle",

abstract = "The common approaches to feature extraction in speech processing are generative and parametric although they are highly sensitive to violations of their model assumptions. Here, we advocate the non-parametric Information Bottleneck (IB). IB is an information theoretic approach that extends minimal sufficient statistics. However, unlike minimal sufficient statistics which does not allow any relevant data loss, IB method enables a principled tradeoff between compactness and the amount of target-related information. IB's ability to improve a broad range of recognition tasks is illustrated for model dimension reduction tasks for speaker recognition and model clustering for age-group verification.",

keywords = "Information bottleneck method, information theory, speaker recognition, speech recognition",

author = "Hecht, {Ron M.} and Elad Noor and Gil Dobry and Yaniv Zigel and Aharon Bar-Hillel and Naftali Tishby",

year = "2013",

month = may,

day = "22",

doi = "10.1109/TASL.2013.2253097",

language = "English",

volume = "21",

pages = "1755--1759",

journal = "IEEE Transactions on Audio, Speech and Language Processing",

issn = "1558-7916",

publisher = "Institute of Electrical and Electronics Engineers",

number = "8",

}

TY - JOUR

T1 - Effective model representation by information bottleneck principle

AU - Hecht, Ron M.

AU - Noor, Elad

AU - Dobry, Gil

AU - Zigel, Yaniv

AU - Bar-Hillel, Aharon

AU - Tishby, Naftali

PY - 2013/5/22

Y1 - 2013/5/22

N2 - The common approaches to feature extraction in speech processing are generative and parametric although they are highly sensitive to violations of their model assumptions. Here, we advocate the non-parametric Information Bottleneck (IB). IB is an information theoretic approach that extends minimal sufficient statistics. However, unlike minimal sufficient statistics which does not allow any relevant data loss, IB method enables a principled tradeoff between compactness and the amount of target-related information. IB's ability to improve a broad range of recognition tasks is illustrated for model dimension reduction tasks for speaker recognition and model clustering for age-group verification.

AB - The common approaches to feature extraction in speech processing are generative and parametric although they are highly sensitive to violations of their model assumptions. Here, we advocate the non-parametric Information Bottleneck (IB). IB is an information theoretic approach that extends minimal sufficient statistics. However, unlike minimal sufficient statistics which does not allow any relevant data loss, IB method enables a principled tradeoff between compactness and the amount of target-related information. IB's ability to improve a broad range of recognition tasks is illustrated for model dimension reduction tasks for speaker recognition and model clustering for age-group verification.

KW - Information bottleneck method

KW - information theory

KW - speaker recognition

KW - speech recognition

UR - http://www.scopus.com/inward/record.url?scp=84877868358&partnerID=8YFLogxK

U2 - 10.1109/TASL.2013.2253097

DO - 10.1109/TASL.2013.2253097

M3 - Article

AN - SCOPUS:84877868358

SN - 1558-7916

VL - 21

SP - 1755

EP - 1759

JO - IEEE Transactions on Audio, Speech and Language Processing

JF - IEEE Transactions on Audio, Speech and Language Processing

IS - 8

M1 - 6480793

ER -

Effective model representation by information bottleneck principle

Abstract

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this