TY - GEN

T1 - Adaptive metric dimensionality reduction

AU - Gottlieb, Lee Ad

AU - Kontorovich, Aryeh

AU - Krauthgamer, Robert

PY - 2013/11/18

Y1 - 2013/11/18

N2 - We study data-adaptive dimensionality reduction in the context of supervised learning in general metric spaces. Our main statistical contribution is a generalization bound for Lipschitz functions in metric spaces that are doubling, or nearly doubling, which yields a new theoretical explanation for empirically reported improvements gained by preprocessing Euclidean data by PCA (Principal Components Analysis) prior to constructing a linear classifier. On the algorithmic front, we describe an analogue of PCA for metric spaces, namely an efficient procedure that approximates the data's intrinsic dimension, which is often much lower than the ambient dimension. Our approach thus leverages the dual benefits of low dimensionality: (1) more efficient algorithms, e.g., for proximity search, and (2) more optimistic generalization bounds.

AB - We study data-adaptive dimensionality reduction in the context of supervised learning in general metric spaces. Our main statistical contribution is a generalization bound for Lipschitz functions in metric spaces that are doubling, or nearly doubling, which yields a new theoretical explanation for empirically reported improvements gained by preprocessing Euclidean data by PCA (Principal Components Analysis) prior to constructing a linear classifier. On the algorithmic front, we describe an analogue of PCA for metric spaces, namely an efficient procedure that approximates the data's intrinsic dimension, which is often much lower than the ambient dimension. Our approach thus leverages the dual benefits of low dimensionality: (1) more efficient algorithms, e.g., for proximity search, and (2) more optimistic generalization bounds.

UR - http://www.scopus.com/inward/record.url?scp=84887503622&partnerID=8YFLogxK

U2 - 10.1007/978-3-642-40935-6_20

DO - 10.1007/978-3-642-40935-6_20

M3 - Conference contribution

AN - SCOPUS:84887503622

SN - 9783642409349

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 279

EP - 293

BT - Algorithmic Learning Theory - 24th International Conference, ALT 2013, Proceedings

T2 - 24th International Conference on Algorithmic Learning Theory, ALT 2013

Y2 - 6 October 2013 through 9 October 2013

ER -