On the Sample Complexity of Privately Learning Axis-Aligned Rectangles

Menachem Sadigurschi; Uri Stemmer

On the Sample Complexity of Privately Learning Axis-Aligned Rectangles

Menachem Sadigurschi, Uri Stemmer

Department of Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Scopus citations

Abstract

We revisit the fundamental problem of learning Axis-Aligned-Rectangles over a finite grid X^d ⊆ R^d with differential privacy. Existing results show that the sample complexity of this problem is at most min { d· log |X|, d^1.5· (log^∗ |X|)^1.5}. That is, existing constructions either require sample complexity that grows linearly with log |X|, or else it grows super linearly with the dimension d. We present a novel algorithm that reduces the sample complexity to only Õ ( d· (log^∗ |X|)^1.5), attaining a dimensionality optimal dependency without requiring the sample complexity to grow with log |X|. The technique used in order to attain this improvement involves the deletion of “exposed” data-points on the go, in a fashion designed to avoid the cost of the adaptive composition theorems. The core of this technique may be of individual interest, introducing a new method for constructing statistically-efficient private algorithms.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021
Editors	Marc'Aurelio Ranzato, Alina Beygelzimer, Yann Dauphin, Percy S. Liang, Jenn Wortman Vaughan
Publisher	Neural information processing systems foundation
Pages	28286-28297
Number of pages	12
ISBN (Electronic)	9781713845393
State	Published - 1 Jan 2021
Event	35th Conference on Neural Information Processing Systems, NeurIPS 2021 - Virtual, Online Duration: 6 Dec 2021 → 14 Dec 2021

Publication series

Name	Advances in Neural Information Processing Systems
Volume	34
ISSN (Print)	1049-5258

Conference

Conference	35th Conference on Neural Information Processing Systems, NeurIPS 2021
City	Virtual, Online
Period	6/12/21 → 14/12/21

ASJC Scopus subject areas

Computer Networks and Communications
Information Systems
Signal Processing

Cite this

Sadigurschi, M., & Stemmer, U. (2021). On the Sample Complexity of Privately Learning Axis-Aligned Rectangles. In MA. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, & J. Wortman Vaughan (Eds.), Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021 (pp. 28286-28297). (Advances in Neural Information Processing Systems; Vol. 34). Neural information processing systems foundation.

Sadigurschi, Menachem ; Stemmer, Uri. / On the Sample Complexity of Privately Learning Axis-Aligned Rectangles. Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021. editor / Marc'Aurelio Ranzato ; Alina Beygelzimer ; Yann Dauphin ; Percy S. Liang ; Jenn Wortman Vaughan. Neural information processing systems foundation, 2021. pp. 28286-28297 (Advances in Neural Information Processing Systems).

@inproceedings{02bef9bce0224c7f83f02dff442048ac,

title = "On the Sample Complexity of Privately Learning Axis-Aligned Rectangles",

abstract = "We revisit the fundamental problem of learning Axis-Aligned-Rectangles over a finite grid Xd ⊆ Rd with differential privacy. Existing results show that the sample complexity of this problem is at most min { d· log |X|, d1.5· (log∗ |X|)1.5}. That is, existing constructions either require sample complexity that grows linearly with log |X|, or else it grows super linearly with the dimension d. We present a novel algorithm that reduces the sample complexity to only {\~O} ( d· (log∗ |X|)1.5), attaining a dimensionality optimal dependency without requiring the sample complexity to grow with log |X|. The technique used in order to attain this improvement involves the deletion of “exposed” data-points on the go, in a fashion designed to avoid the cost of the adaptive composition theorems. The core of this technique may be of individual interest, introducing a new method for constructing statistically-efficient private algorithms.",

author = "Menachem Sadigurschi and Uri Stemmer",

note = "Publisher Copyright: {\textcopyright} 2021 Neural information processing systems foundation. All rights reserved.; 35th Conference on Neural Information Processing Systems, NeurIPS 2021 ; Conference date: 06-12-2021 Through 14-12-2021",

year = "2021",

month = jan,

day = "1",

language = "English",

series = "Advances in Neural Information Processing Systems",

publisher = "Neural information processing systems foundation",

pages = "28286--28297",

editor = "Marc'Aurelio Ranzato and Alina Beygelzimer and Yann Dauphin and Liang, {Percy S.} and {Wortman Vaughan}, Jenn",

booktitle = "Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021",

}

Sadigurschi, M & Stemmer, U 2021, On the Sample Complexity of Privately Learning Axis-Aligned Rectangles. in MA Ranzato, A Beygelzimer, Y Dauphin, PS Liang & J Wortman Vaughan (eds), Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021. Advances in Neural Information Processing Systems, vol. 34, Neural information processing systems foundation, pp. 28286-28297, 35th Conference on Neural Information Processing Systems, NeurIPS 2021, Virtual, Online, 6/12/21.

On the Sample Complexity of Privately Learning Axis-Aligned Rectangles. / Sadigurschi, Menachem; Stemmer, Uri.
Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021. ed. / Marc'Aurelio Ranzato; Alina Beygelzimer; Yann Dauphin; Percy S. Liang; Jenn Wortman Vaughan. Neural information processing systems foundation, 2021. p. 28286-28297 (Advances in Neural Information Processing Systems; Vol. 34).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - On the Sample Complexity of Privately Learning Axis-Aligned Rectangles

AU - Sadigurschi, Menachem

AU - Stemmer, Uri

PY - 2021/1/1

Y1 - 2021/1/1

N2 - We revisit the fundamental problem of learning Axis-Aligned-Rectangles over a finite grid Xd ⊆ Rd with differential privacy. Existing results show that the sample complexity of this problem is at most min { d· log |X|, d1.5· (log∗ |X|)1.5}. That is, existing constructions either require sample complexity that grows linearly with log |X|, or else it grows super linearly with the dimension d. We present a novel algorithm that reduces the sample complexity to only Õ ( d· (log∗ |X|)1.5), attaining a dimensionality optimal dependency without requiring the sample complexity to grow with log |X|. The technique used in order to attain this improvement involves the deletion of “exposed” data-points on the go, in a fashion designed to avoid the cost of the adaptive composition theorems. The core of this technique may be of individual interest, introducing a new method for constructing statistically-efficient private algorithms.

AB - We revisit the fundamental problem of learning Axis-Aligned-Rectangles over a finite grid Xd ⊆ Rd with differential privacy. Existing results show that the sample complexity of this problem is at most min { d· log |X|, d1.5· (log∗ |X|)1.5}. That is, existing constructions either require sample complexity that grows linearly with log |X|, or else it grows super linearly with the dimension d. We present a novel algorithm that reduces the sample complexity to only Õ ( d· (log∗ |X|)1.5), attaining a dimensionality optimal dependency without requiring the sample complexity to grow with log |X|. The technique used in order to attain this improvement involves the deletion of “exposed” data-points on the go, in a fashion designed to avoid the cost of the adaptive composition theorems. The core of this technique may be of individual interest, introducing a new method for constructing statistically-efficient private algorithms.

UR - http://www.scopus.com/inward/record.url?scp=85131882514&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:85131882514

T3 - Advances in Neural Information Processing Systems

SP - 28286

EP - 28297

BT - Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021

A2 - Ranzato, Marc'Aurelio

A2 - Beygelzimer, Alina

A2 - Dauphin, Yann

A2 - Liang, Percy S.

A2 - Wortman Vaughan, Jenn

PB - Neural information processing systems foundation

T2 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021

Y2 - 6 December 2021 through 14 December 2021

ER -

Sadigurschi M, Stemmer U. On the Sample Complexity of Privately Learning Axis-Aligned Rectangles. In Ranzato MA, Beygelzimer A, Dauphin Y, Liang PS, Wortman Vaughan J, editors, Advances in Neural Information Processing Systems 34 - 35th Conference on Neural Information Processing Systems, NeurIPS 2021. Neural information processing systems foundation. 2021. p. 28286-28297. (Advances in Neural Information Processing Systems).

On the Sample Complexity of Privately Learning Axis-Aligned Rectangles

Abstract

Publication series

Conference

ASJC Scopus subject areas

Other files and links

Fingerprint

Cite this