Efficient regression in metric spaces via approximate Lipschitz extension

Lee Ad Gottlieb, Aryeh Kontorovich, Robert Krauthgamer

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

We present a framework for performing efficient regression in general metric spaces. Roughly speaking, our regressor predicts the value at a new point by computing a Lipschitz extension - the smoothest function consistent with the observed data - while performing an optimized structural risk minimization to avoid overfitting. The offline (learning) and online (inference) stages can be solved by convex programming, but this naive approach has runtime complexity O(n3), which is prohibitive for large datasets. We design instead an algorithm that is fast when the doubling dimension, which measures the "intrinsic" dimensionality of the metric space, is low. We make dual use of the doubling dimension: first, on the statistical front, to bound fat-shattering dimension of the class of Lipschitz functions (and obtain risk bounds); and second, on the computational front, to quickly compute a hypothesis function and a prediction based on Lipschitz extension. Our resulting regressor is both asymptotically strongly consistent and comes with finite-sample risk bounds, while making minimal structural and noise assumptions.

Original languageEnglish
Title of host publicationSimilarity-Based Pattern Recognition - Second International Workshop, SIMBAD 2013, Proceedings
Pages43-58
Number of pages16
DOIs
StatePublished - 12 Jul 2013
Event2nd International Workshop on Similarity-Based Pattern Analysis and Recognition, SIMBAD 2013 - York, United Kingdom
Duration: 3 Jul 20135 Jul 2013

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume7953 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference2nd International Workshop on Similarity-Based Pattern Analysis and Recognition, SIMBAD 2013
Country/TerritoryUnited Kingdom
CityYork
Period3/07/135/07/13

Keywords

  • convex program
  • metric space
  • regression

Fingerprint

Dive into the research topics of 'Efficient regression in metric spaces via approximate Lipschitz extension'. Together they form a unique fingerprint.

Cite this