TY - GEN

T1 - Simultaneous private learning of multiple concepts

AU - Bun, Mark

AU - Nissim, Kobbi

AU - Stemmer, Uri

PY - 2016/1/14

Y1 - 2016/1/14

N2 - We investigate the direct-sum problem in the context of differentially private PAC learning: What is the sample complexity of solving k learning tasks simultaneously under differential privacy, and how does this cost compare to that of solving k learning tasks without privacy? In our setting, an individual example consists of a domain element x labeled by k unknown concepts (c1; ck). The goal of a multi-learner is to output k hypotheses (h1; hk) that generalize the input examples. Without concern for privacy, the sample complexity needed to simultaneously learn k concepts is essentially the same as needed for learning a single concept. Under differential privacy, the basic strategy of learning each hypothesis independently yields sample complexity that grows polynomially with k. For some concept classes, we give multi-learners that require fewer samples than the basic strategy. Unfortunately, however, we also give lower bounds showing that even for very simple concept classes, the sample cost of private multi-learning must grow polynomially in k.

AB - We investigate the direct-sum problem in the context of differentially private PAC learning: What is the sample complexity of solving k learning tasks simultaneously under differential privacy, and how does this cost compare to that of solving k learning tasks without privacy? In our setting, an individual example consists of a domain element x labeled by k unknown concepts (c1; ck). The goal of a multi-learner is to output k hypotheses (h1; hk) that generalize the input examples. Without concern for privacy, the sample complexity needed to simultaneously learn k concepts is essentially the same as needed for learning a single concept. Under differential privacy, the basic strategy of learning each hypothesis independently yields sample complexity that grows polynomially with k. For some concept classes, we give multi-learners that require fewer samples than the basic strategy. Unfortunately, however, we also give lower bounds showing that even for very simple concept classes, the sample cost of private multi-learning must grow polynomially in k.

KW - Agnostic learning

KW - Differential privacy

KW - Directsum

KW - PAC learning

UR - http://www.scopus.com/inward/record.url?scp=84966534247&partnerID=8YFLogxK

U2 - 10.1145/2840728.2840747

DO - 10.1145/2840728.2840747

M3 - Conference contribution

AN - SCOPUS:84966534247

T3 - ITCS 2016 - Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science

SP - 369

EP - 380

BT - ITCS 2016 - Proceedings of the 2016 ACM Conference on Innovations in Theoretical Computer Science

PB - Association for Computing Machinery, Inc

T2 - 7th ACM Conference on Innovations in Theoretical Computer Science, ITCS 2016

Y2 - 14 January 2016 through 16 January 2016

ER -