Tight sample complexity of large-margin learning

Sivan Sabato; Nathan Srebro; Naftali Tishby

Tight sample complexity of large-margin learning

Sivan Sabato, Nathan Srebro, Naftali Tishby

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Scopus citations

Abstract

We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L₂ regularization: We introduce the γ-adapted-dimension, which is a simple function of the spectrum of a distribution's covariance matrix, and show distribution-specific upper and lower bounds on the sample complexity, both governed by the γ-adapted-dimension of the source distribution. We conclude that this new quantity tightly characterizes the true sample complexity of large-margin classification. The bounds hold for a rich family of sub-Gaussian distributions.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 23
Subtitle of host publication	24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010
Publisher	Neural Information Processing Systems
ISBN (Print)	9781617823800
State	Published - 1 Jan 2010
Externally published	Yes
Event	24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010 - Vancouver, BC, Canada Duration: 6 Dec 2010 → 9 Dec 2010

Conference

Conference	24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010
Country/Territory	Canada
City	Vancouver, BC
Period	6/12/10 → 9/12/10

ASJC Scopus subject areas

Information Systems

Cite this

@inproceedings{67a06e2481ef4d7b9ace72b6c1282f69,

title = "Tight sample complexity of large-margin learning",

abstract = "We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the γ-adapted-dimension, which is a simple function of the spectrum of a distribution's covariance matrix, and show distribution-specific upper and lower bounds on the sample complexity, both governed by the γ-adapted-dimension of the source distribution. We conclude that this new quantity tightly characterizes the true sample complexity of large-margin classification. The bounds hold for a rich family of sub-Gaussian distributions.",

author = "Sivan Sabato and Nathan Srebro and Naftali Tishby",

year = "2010",

month = jan,

day = "1",

language = "English",

isbn = "9781617823800",

booktitle = "Advances in Neural Information Processing Systems 23",

publisher = "Neural Information Processing Systems",

note = "24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010 ; Conference date: 06-12-2010 Through 09-12-2010",

}

Sabato, S, Srebro, N & Tishby, N 2010, Tight sample complexity of large-margin learning. in Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010. Neural Information Processing Systems, 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010, Vancouver, BC, Canada, 6/12/10.

Tight sample complexity of large-margin learning. / Sabato, Sivan; Srebro, Nathan; Tishby, Naftali.
Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010. Neural Information Processing Systems, 2010.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Tight sample complexity of large-margin learning

AU - Sabato, Sivan

AU - Srebro, Nathan

AU - Tishby, Naftali

PY - 2010/1/1

Y1 - 2010/1/1

N2 - We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the γ-adapted-dimension, which is a simple function of the spectrum of a distribution's covariance matrix, and show distribution-specific upper and lower bounds on the sample complexity, both governed by the γ-adapted-dimension of the source distribution. We conclude that this new quantity tightly characterizes the true sample complexity of large-margin classification. The bounds hold for a rich family of sub-Gaussian distributions.

AB - We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the γ-adapted-dimension, which is a simple function of the spectrum of a distribution's covariance matrix, and show distribution-specific upper and lower bounds on the sample complexity, both governed by the γ-adapted-dimension of the source distribution. We conclude that this new quantity tightly characterizes the true sample complexity of large-margin classification. The bounds hold for a rich family of sub-Gaussian distributions.

UR - http://www.scopus.com/inward/record.url?scp=85161993206&partnerID=8YFLogxK

M3 - Conference contribution

SN - 9781617823800

BT - Advances in Neural Information Processing Systems 23

PB - Neural Information Processing Systems

T2 - 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010

Y2 - 6 December 2010 through 9 December 2010

ER -

Tight sample complexity of large-margin learning

Abstract

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this