Signal Processing and Machine Learning on Manifolds, with Applications to Invariant Detection

Inspired by the ideas of Representation Theory, this report contains a comprehensive account on applications of this general theory to group—invariant signal processing and machine learning on manifolds, expressed through the derivation of several approaches to invariant classification of images and 3D point clouds as well as to covariant estimation of coordinate transformations, based on the Universal Manifold Embedding.

Define a group actzbn, especially a group action on coordinates, to model signal or image transformations. The group acts on the function space by acting on its domain. We then define an equivalence class (an orbit) of signals that may be reached with a group action on any other member of the equivalence class. The inference problems are to estimate the group action that moves any image on an orbit to any other image on the orbit, and to identify observations that lie on the same orbit; this is eqmi/(z7'7I(i'rLt (or co'u(i'r'iant) estz'mat'/Ian and in’ua7'7Ia7Lt detecti07I,. The key is to find a G—eqm'va7'7Iunt map on the signal space, namely one that commutes with the group action on coordinates; these maps serve as covariant statistics for estimation of group action, and from them invariant statistics may be derived for detection. In this report, these general concepts are made concrete, through different aspects in the implementation of the Universal Manifold Embedding (UME) for 2D and 3D observations:

1. In Matched Manifold Detection for Group-Invariant Registration and Classification of Images we consider the set of possible observations turned out by geometric and radiometric transformations of an object. This set is generally a manifold in the ambient space of observations. It has been shown that in those cases where the geometric deformations are affine and the radiometric deformations are monotonic, the radiometry invariant universal manifold embedding (RIUME) provides a mapping from the orbit of deformed observations to a single low dimensional linear subspace of Euclidean space. This linear subspace is invariant to the geometric and radiometric transformations and hence is a representative of the orbit. It thus naturally serves as an invariant statistic for solving problems of joint transformation estimation and detection or classification. In the unsupervised detection problem, subspaces evaluated from two observations are tested for the similarity of the observed object and their relative transformation is estimated from the RIUME matrix representation. In the classification set—up the RIUME subspace extracted from an experimental observation is tested against a set of subspaces representing the different object manifolds, in search for the nearest class. We show how to extract a set of mutually orthogonal subspaces, where each subspace represents a different object manifold. In the presence of observation noise, the observations do not lie strictly on the manifold and the resulting RIUME subspaces are noisy. We derive a method for estimating the mean subspace representation of a manifold of deformed observations. To optimize the performance of the matched manifold detector in the presence of observation noise, an analytic solution for choosing the RIUME nonlinear operators is derived, achieving the effect of simultaneous denoising of the object manifolds. The invariant representation of the object is the basis of a matched manifold detection and tracking framework for objects that undergo complex geometric and radiometric deformations. The experimental results on natural scenes demonstrate the generality and applicability of the RIUME framework for classification, detection, and dense registration;

2. In Estimating Rigid Transformations of Point Clouds Using the Universal Manifold Embedding we present a closed form solution to the problem of registration and detection of 3-D point clouds undergoing unknown rigid transformations. The solution is obtained by adapting the general framework of the universal manifold embedding (UME) to the case where the transformations the object may undergo are rigid. The UME nonlinearly maps functions related by certain types of geometric transformations of coordinates to the same linear subspace of some Euclidean space while retaining the information required to recover the transformation. Therefore registration, matching and classification can be solved as linear problems in a low dimensional linear space. While a variety of methods exist for point cloud registration, the Rigid Transformation Universal Manifold Embedding (RTUME) derived here is notably different as registration is achieved by a closed form solution that employs the UME low dimensional representation of the shapes to be registered.

In this context we present in Dual Transformation and Manifold Distances Voting for Outlier Rejection in Point Cloud Registration a novel outlier rejection scheme for point cloud registration using SE(3) voting on local transformation estimates with a dual consensus constraint. Each putative matching pair of points is equipped with a local transformation estimate using the Rigid Transformation Universal Manifold Embedding. Putative matching pairs with similar local estimates are then clustered together and the global transformation between point clouds is estimated for each cluster. Finally, the cluster with the majority of the votes such that the average of local transformations agrees with its associated global transformation is selected for completing the registration. This approach successfully deals with up to 99.5% outliers, where state of the art fails.

3. In Grassmannian Dimensionality Reduction for Optimized Universal Manifold Embedding Representation of 3D Point Clouds we further consider 3-D objects and the orbit of equivalent objects turned out by their rigid transformations. We clarify the way in which level-set images, computed at each quantization level in an observation, serve as a basis for the invariant subspaces in RTUME. In the presence of observation noise and random sampling patterns of the point clouds, the observations do not lie strictly on the manifold and the resulting RTUME subspaces are noisy. Inspired by the ideas of Locality Preserving Projections and Grassmannian dimensionality reduction, we derive an optimal companding of the level-set images yielding the Grassmannian dimensionality reduction universal manifold embedding (GDRUME). We evaluate the proposed method in a classification task of noisy point clouds and compare its performance to that of the state-of-the-art PointNet classification DNN. We show that in the presence of noise, GDRUME provides highly accurate classification results, while the performance of PointNet is poor.

4. In DeepUME: Learning the Universal Manifold Embedding for Robust Point Cloud Registration we approach registration of under-sampled and noisy 3D point clouds with a fusion of the Universal Manifold Embedding (UME) method and an unsupervised deep neural network. In order to overcome a major obstacle in the learning process under full rotation range, we employ an SO(3)-invariant coordinate system to learn SO(3)-invariant features, later to be utilized by the closed-form geometric UME method for transformation estimation. Finally, we show that our hybrid method outperforms state-of-the-art registration methods in various scenarios, and generalizes well to unseen datasets.