Highly Overparameterized Optical Flow Using PatchMatch Belief Propagation

Michael Hornáek¹, Frederic Besse², Jan Kautz², Andrew Fitzgibbon³, and Carsten Rother⁴

¹TU Vienna, Austria
michael.hornacek@tuwien.ac.at

²University College London, UK
f.besse@cs.ucl.ac.uk
j.kautz@cs.ucl.ac.uk

³Microsoft Research Cambridge, UK
awf@microsoft.com

⁴TU Dresden, Germany
carsten.rother@tu-dresden.de

Abstract. Motion in the image plane is ultimately a function of 3D motion in space. We propose to compute optical flow using what is ostensibly an extreme overparameterization: depth, surface normal, and frame-to-frame 3D rigid body motion at every pixel, giving a total of 9 DoF. The advantages of such an overparameterization are twofold: first, geometrically meaningful reasoning can be called upon in the optimization, reflecting possible 3D motion in the underlying scene; second, the ‘fronto-parallel’ assumption implicit in the use of traditional matching pixel windows is ameliorated because the parameterization determines a plane-induced homography at every pixel. We show that optimization over this high-dimensional, continuous state space can be carried out using an adaptation of the recently introduced PatchMatch Belief Propagation (PMBP) energy minimization algorithm, and that the resulting flow fields compare favorably to the state of the art on a number of small- and large-displacement datasets.

Keywords: Optical flow, large displacement, 9 DoF, PatchMatch, PMBP

LNCS 8691, p. 220 ff.

Full article in PDF | BibTeX

lncs@springer.com
© Springer International Publishing Switzerland 2014