Boosting Conditional Probability Estimators

Dan Gutfreund, Aryeh Kontorovich, Ran Levy, Michal Rosen-Zvi

Research output: Contribution to conferencePaperpeer-review

Abstract

In the standard agnostic multiclass model, <instance, label> pairs are sampled independently from some underlying distribution. This distribution induces a conditional probability over the labels given an instance, and our goal in this paper is to learn this conditional distribution. Since even unconditional densities are quite challenging to learn, we give our learner access to <instance, conditional distribution> pairs. Assuming a base learner oracle in this model, we might seek a boosting algorithm for constructing a strong learner. Unfortunately, without further assumptions, this is provably impossible. However, we give a new boosting algorithm that succeeds in the following sense: given a base learner guaranteed to achieve some average accuracy (i.e., risk), we efficiently construct a learner that achieves the same level of accuracy with arbitrarily high probability. We give generalization guarantees of several different kinds, including distribution-free accuracy and risk bounds. None of our estimates depend on the number of boosting rounds and some of them admit dimension-free formulations.

Original languageEnglish
StatePublished - 2014
Event2014 International Symposium on Artificial Intelligence and Mathematics, ISAIM 2014 - Fort Lauderdale, United States
Duration: 6 Jan 20148 Jan 2014

Conference

Conference2014 International Symposium on Artificial Intelligence and Mathematics, ISAIM 2014
Country/TerritoryUnited States
CityFort Lauderdale
Period6/01/148/01/14

ASJC Scopus subject areas

  • Applied Mathematics
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'Boosting Conditional Probability Estimators'. Together they form a unique fingerprint.

Cite this