Universal Bayes Consistency in Metric Spaces

Steve Hanneke, Aryeh Kontorovich, Sivan Sabato, Roi Weiss

Research output: Chapter in Book/Report/Conference proceedingConference contribution

4 Scopus citations


We show that a recently proposed 1-nearest-neighbor-based multiclass learning algorithm is universally strongly Bayes consistent in all metric spaces where such Bayes consistency is possible, making it an "optimistically universal"Bayes-consistent learner. This is the first learning algorithm known to enjoy this property; by comparison, k-NN and its variants are not generally universally Bayes consistent, except under additional structural assumptions, such as an inner product, a norm, finite doubling dimension, or a Besicovitch-type property.The metric spaces in which universal Bayes consistency is possible are the "essentially separable"ones-a new notion that we define, which is more general than standard separability. The existence of metric spaces that are not essentially separable is independent of the ZFC axioms of set theory. We prove that essential separability exactly characterizes the existence of a universal Bayes-consistent learner for the given metric space. In particular, this yields the first impossibility result for universal Bayes consistency.Taken together, these positive and negative results resolve the open problems posed in Kontorovich, Sabato, Weiss (2017).

Original languageEnglish
Title of host publication2020 Information Theory and Applications Workshop, ITA 2020
PublisherInstitute of Electrical and Electronics Engineers
ISBN (Electronic)9781728141909
StatePublished - 2 Feb 2020
Event2020 Information Theory and Applications Workshop, ITA 2020 - San Diego, United States
Duration: 2 Feb 20207 Feb 2020


Conference2020 Information Theory and Applications Workshop, ITA 2020
Country/TerritoryUnited States
CitySan Diego


  • Bayes consistency
  • classification
  • metric space
  • nearest neighbor

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computational Theory and Mathematics
  • Computer Science Applications
  • Information Systems and Management
  • Control and Optimization


Dive into the research topics of 'Universal Bayes Consistency in Metric Spaces'. Together they form a unique fingerprint.

Cite this