Highly accurate matching of weakly localized features

Matching corresponding local patches between images is a fundamental building block in many computer vision algorithms. It is extensively used in applications like three-dimensional structure estimation, model fitting, superresolution, image retrieval, and image recognition. As opposed to global image registration approaches, local matching reduces the high-dimensional challenge of recovering geometric relations between images to a series of relatively simple and independent tasks, defined in a limited local context. This approach is geometrically very flexible, simple to model, and has clear computational advantages. But it also has two fundamental practical shortcomings: (1) sparsity: The need to rely on repeatable features for matching limits the potential coverage of the scene to highly textured locations; (2) reliability: The limited spatial context in which those methods work often does not contain enough information for achieving reliable matches. These shortcomings also tend to trade off. While highly textured features, such as corners and blobs, often produce reliable matches, they are also relatively rare and thus very sparse in the image. And conversely, more common textures, such as edges or ridges, are largely discarded due to their low reliability for matching. We observe that while classic methods avoided using poorly localized features (e.g., edges) as matching candidates, these features contain highly valuable information for matching. We show how, given the appropriate geometric context, reliable matches can be produced from these features, contributing to a better coverage of the scene, while also producing highly accurate geometric transformation estimation. We present a statistically attractive framework for encoding the uncertainty that stems from using these features into a coupled geometric estimation and match extraction process. We examine the practical application of the proposed framework to the problems of guided matching and affine region expansion and show significant improvement over preceding methods. We compare our approach to state-of-the-art point matching methods and show its attractiveness in a variety of geometrically challenging scenarios.