Low cost CMOS cameras can have an acquisition mode called rolling shutter which sequentially exposes the scan-lines. When a single object moves with respect to the camera, this creates image distortions. Assuming 2D-3D correspondences known, previous work showed that the object pose and kinematics can be estimated from a single rolling shutter image. This was achieved using a suboptimal initialization followed by local iterative optimization.
We propose a polynomial projection model for rolling shutter cameras and a constrained global optimization of its parameters. This is done by means of a semidefinite programming problem obtained from the generalized problem of moments method. Contrarily to previous work, our optimization does not require an initialization and ensures that the global minimum is achieved. This allows us to build automatically robust 2D-3D correspondences using a template to provide an initial set of correspondences.
Experiments show that our method slightly improves previous work on both simulated and real data. This is due to local minima into which previous methods get trapped. We also successfully experimented building 2D-3D correspondences automatically with both simulated and real data.
Reference
Prisacariu, Victor A., and Ian D. Reid."Pwp3d: Real-time segmentation and tracking of 3d objects." International journal of computer vision 98.3(2012): 335-354.
Active Vision Laboratory, Department of Engineering Science, University of Oxford
Tjaden, Henning, Ulrich Schwanecke, and Elmar Schömer."Real-Time Monocular Segmentation and Pose Tracking of Multiple Objects." European Conference on Computer Vision. Springer International Publishing, 2016.
Computer Science Department, RheinMain University of Applied Sciences
Institute of Computer Science, Johannes Gutenberg University Mainz
2016-09-27