Stable Sample Compression Schemes: New Applications and an Optimal SVM Margin Bound

Steve Hanneke; Aryeh Kontorovich

Stable Sample Compression Schemes: New Applications and an Optimal SVM Margin Bound

Steve Hanneke, Aryeh Kontorovich

Department of Computer Science

Research output: Contribution to journal › Conference article › peer-review

16 Scopus citations

Abstract

We analyze a family of supervised learning algorithms based on sample compression schemes that are stable, in the sense that removing points from the training set which were not selected for the compression set does not alter the resulting classifier. We use this technique to derive a variety of novel or improved data-dependent generalization bounds for several learning algorithms. In particular, we prove a new margin bound for SVM, removing a log factor. The new bound is provably optimal. This resolves a long-standing open question about the PAC margin bounds achievable by SVM.

Original language	English
Pages (from-to)	697-721
Number of pages	25
Journal	Proceedings of Machine Learning Research
Volume	132
State	Published - 1 Jan 2021
Event	32nd International Conference on Algorithmic Learning Theory, ALT 2021 - Virtual, Online Duration: 16 Mar 2021 → 19 Mar 2021

Keywords

margin
sample compression
support vector machines

ASJC Scopus subject areas

Artificial Intelligence
Software
Control and Systems Engineering
Statistics and Probability

Cite this

@article{621c296cb0d4483aa2d6a57fa17aeaea,

title = "Stable Sample Compression Schemes: New Applications and an Optimal SVM Margin Bound",

abstract = "We analyze a family of supervised learning algorithms based on sample compression schemes that are stable, in the sense that removing points from the training set which were not selected for the compression set does not alter the resulting classifier. We use this technique to derive a variety of novel or improved data-dependent generalization bounds for several learning algorithms. In particular, we prove a new margin bound for SVM, removing a log factor. The new bound is provably optimal. This resolves a long-standing open question about the PAC margin bounds achievable by SVM.",

keywords = "margin, sample compression, support vector machines",

author = "Steve Hanneke and Aryeh Kontorovich",

note = "Publisher Copyright: {\textcopyright} 2021 S. Hanneke & A. Kontorovich.; 32nd International Conference on Algorithmic Learning Theory, ALT 2021 ; Conference date: 16-03-2021 Through 19-03-2021",

year = "2021",

month = jan,

day = "1",

language = "English",

volume = "132",

pages = "697--721",

journal = "Proceedings of Machine Learning Research",

issn = "2640-3498",

}

TY - JOUR

T1 - Stable Sample Compression Schemes

T2 - 32nd International Conference on Algorithmic Learning Theory, ALT 2021

AU - Hanneke, Steve

AU - Kontorovich, Aryeh

PY - 2021/1/1

Y1 - 2021/1/1

N2 - We analyze a family of supervised learning algorithms based on sample compression schemes that are stable, in the sense that removing points from the training set which were not selected for the compression set does not alter the resulting classifier. We use this technique to derive a variety of novel or improved data-dependent generalization bounds for several learning algorithms. In particular, we prove a new margin bound for SVM, removing a log factor. The new bound is provably optimal. This resolves a long-standing open question about the PAC margin bounds achievable by SVM.

AB - We analyze a family of supervised learning algorithms based on sample compression schemes that are stable, in the sense that removing points from the training set which were not selected for the compression set does not alter the resulting classifier. We use this technique to derive a variety of novel or improved data-dependent generalization bounds for several learning algorithms. In particular, we prove a new margin bound for SVM, removing a log factor. The new bound is provably optimal. This resolves a long-standing open question about the PAC margin bounds achievable by SVM.

KW - margin

KW - sample compression

KW - support vector machines

UR - http://www.scopus.com/inward/record.url?scp=85103809787&partnerID=8YFLogxK

M3 - Conference article

AN - SCOPUS:85103809787

SN - 2640-3498

VL - 132

SP - 697

EP - 721

JO - Proceedings of Machine Learning Research

JF - Proceedings of Machine Learning Research

Y2 - 16 March 2021 through 19 March 2021

ER -

Stable Sample Compression Schemes: New Applications and an Optimal SVM Margin Bound

Abstract

Keywords

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this