Gradient Surfing: A New Deterministic Approach for Low-Dimensional Global Optimization

Efrat Taig, Ohad Ben-Shahar

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

We describe a novel global optimization technique which utilizes global minima basins of attraction in order to quickly converge to a global minima. A key to the proposed method is the “steeper goes deeper” heuristic: coupling between magnitudes of gradients on different level sets of a basin of attraction and the depth of its minima. Local minima are avoided with a combination of local optimization and a heuristic-based leaping step. Gradient surfing performance is evaluated across a set of small-scale problems from the literature, and results are compared to those of 12 previously published methods. A practical six-dimensional non-convex image registration application is presented as well, where GS performance exceeds that of classic global optimization methods in both speed and accuracy. Additionally, we validate the optimization method by applying a new Gaussian mixture model benchmark for non-convex function. Finally, the “steeper goes deeper” heuristic is validated empirically on five different classes of non-convex functions using two different evaluation approaches. In all cases, steeper gradients are shown to lead to deeper optima with a high probability.

Original languageEnglish
Pages (from-to)855-878
Number of pages24
JournalJournal of Optimization Theory and Applications
Volume180
Issue number3
DOIs
StatePublished - 15 Mar 2019

Keywords

  • Basin of attraction
  • Global optimization
  • Gradient descent
  • Level set
  • Local optimization

ASJC Scopus subject areas

  • Control and Optimization
  • Management Science and Operations Research
  • Applied Mathematics

Fingerprint

Dive into the research topics of 'Gradient Surfing: A New Deterministic Approach for Low-Dimensional Global Optimization'. Together they form a unique fingerprint.

Cite this