TY - JOUR

T1 - Boosting conditional probability estimators

AU - Gutfreund, Dan

AU - Kontorovich, Aryeh

AU - Levy, Ran

AU - Rosen-Zvi, Michal

N1 - Funding Information:
A preliminary version was invited to ISAIM 2014. A.K. was partially supported by the Israel Science Foundation (grant No. 1141/12) and a Yahoo Faculty award.
Publisher Copyright:
© 2015, Springer International Publishing Switzerland.

PY - 2017/3/1

Y1 - 2017/3/1

N2 - In the standard agnostic multiclass model, pairs are sampled independently from some underlying distribution. This distribution induces a conditional probability over the labels given an instance, and our goal in this paper is to learn this conditional distribution. Since even unconditional densities are quite challenging to learn, we give our learner access to pairs. Assuming a base learner oracle in this model, we might seek a boosting algorithm for constructing a strong learner. Unfortunately, without further assumptions, this is provably impossible. However, we give a new boosting algorithm that succeeds in the following sense: given a base learner guaranteed to achieve some average accuracy (i.e., risk), we efficiently construct a learner that achieves the same level of accuracy with arbitrarily high probability. We give generalization guarantees of several different kinds, including distribution-free accuracy and risk bounds. None of our estimates depend on the number of boosting rounds and some of them admit dimension-free formulations.

AB - In the standard agnostic multiclass model, pairs are sampled independently from some underlying distribution. This distribution induces a conditional probability over the labels given an instance, and our goal in this paper is to learn this conditional distribution. Since even unconditional densities are quite challenging to learn, we give our learner access to pairs. Assuming a base learner oracle in this model, we might seek a boosting algorithm for constructing a strong learner. Unfortunately, without further assumptions, this is provably impossible. However, we give a new boosting algorithm that succeeds in the following sense: given a base learner guaranteed to achieve some average accuracy (i.e., risk), we efficiently construct a learner that achieves the same level of accuracy with arbitrarily high probability. We give generalization guarantees of several different kinds, including distribution-free accuracy and risk bounds. None of our estimates depend on the number of boosting rounds and some of them admit dimension-free formulations.

KW - Boosting

KW - Conditional density

UR - http://www.scopus.com/inward/record.url?scp=84930917874&partnerID=8YFLogxK

U2 - 10.1007/s10472-015-9465-7

DO - 10.1007/s10472-015-9465-7

M3 - Article

AN - SCOPUS:84930917874

VL - 79

SP - 129

EP - 144

JO - Annals of Mathematics and Artificial Intelligence

JF - Annals of Mathematics and Artificial Intelligence

SN - 1012-2443

IS - 1-3

ER -