Learning Distance Functions using Equivalence Relations

Aharon Bar-Hillel, Tomer Hertz, Noam Shental, Daphna Weinshall

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

330 Scopus citations

Abstract

We address the problem of learning distance metrics using side-information in the form of groups of "similar" points. We propose to use the RCA algorithm, which is a simple and efficient algorithm for learning a full ranked Mahalanobis metric (Shental et al., 2002). We first show that RCA obtains the solution to an interesting optimization problem, founded on an information theoretic basis. If the Mahalanobis matrix is allowed to be singular, we show that Fisher's linear discriminant followed by RCA is the optimal dimensionality reduction algorithm under the same criterion. We then show how this optimization problem is related to the criterion optimized by another recent algorithm for metric learning (Xing et al., 2002), which uses the same kind of side information. We empirically demonstrate that learning a distance metric using the RCA algorithm significantly improves clustering performance, similarly to the alternative algorithm. Since the RCA algorithm is much more efficient and cost effective than the alternative, as it only uses closed form expressions of the data, it seems like a preferable choice for the learning of full rank Mahalanobis distances.

Original languageEnglish
Title of host publicationProceedings, Twentieth International Conference on Machine Learning
EditorsT. Fawcett, N. Mishra
Pages11-18
Number of pages8
StatePublished - 1 Dec 2003
Externally publishedYes
EventProceedings, Twentieth International Conference on Machine Learning - Washington, DC, United States
Duration: 21 Aug 200324 Aug 2003

Publication series

NameProceedings, Twentieth International Conference on Machine Learning
Volume1

Conference

ConferenceProceedings, Twentieth International Conference on Machine Learning
Country/TerritoryUnited States
CityWashington, DC
Period21/08/0324/08/03

Keywords

  • Clustering
  • Feature selection
  • Learning from partial knowledge
  • Semi-supervised learning

ASJC Scopus subject areas

  • Engineering (all)

Fingerprint

Dive into the research topics of 'Learning Distance Functions using Equivalence Relations'. Together they form a unique fingerprint.

Cite this