Toward robust estimation of specular flow

Specular flow is an important class of optical flow whose utility in visual tasks has gained much interest in contemporary vision research. Unfortunately, however, reliably estimation of specular flow from image sequences is an open question that was never addressed formally before. In this paper we first argue that existing optical flow algorithms are incapable of reliable specular flow estimation due to their typical regularization criteria that conflict the unique and singular structure of specular flows. We show these discrepancies both qualitatively and using quantitative evaluation based on a first-of-its-kind benchmark dataset with ground truth specular flow data. We then suggest to generalize the popular optical flow variational framework using spatial weighting of different regularizers, and we propose new regularization terms that correspond better to the expected singularities of specular flows. Finally, we show how these contributions significantly improves specular flow estimation.