Predictive PAC learning and process decompositions

Cosma Rohilla Shalizi, Aryeh Kontorovich

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

13 Scopus citations

Abstract

We informally call a stochastic process learnable if it admits a generalization error approaching zero in probability for any concept class with finite VC-dimension (IID processes are the simplest example). A mixture of learnable processes need not be learnable itself, and certainly its generalization error need not decay at the same rate. In this paper, we argue that it is natural in predictive PAC to condition not on the past observations but on the mixture component of the sample path. This definition not only matches what a realistic learner might demand, but also allows us to sidestep several otherwise grave problems in learning from dependent data. In particular, we give a novel PAC generalization bound for mixtures of learnable processes with a generalization error that is not worse than that of each mixture component. We also provide a characterization of mixtures of absolutely regular (β-mixing) processes, of independent probability-theoretic interest.

Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems
Volume26
StatePublished - 1 Jan 2013
Event27th Annual Conference on Neural Information Processing Systems, NIPS 2013 - Lake Tahoe, NV, United States
Duration: 5 Dec 201310 Dec 2013

Conference

Conference27th Annual Conference on Neural Information Processing Systems, NIPS 2013
Country/TerritoryUnited States
CityLake Tahoe, NV
Period5/12/1310/12/13

Fingerprint

Dive into the research topics of 'Predictive PAC learning and process decompositions'. Together they form a unique fingerprint.

Cite this