Abstract

The present invention is embodied in systems and methods for determining structure and motion of a three-dimensional (3D) object using two-dimensional (2D) images of the object obtained from multiple sets of views with different projection models, such as from a full perspective view and a weak perspective views. A novel fundamental matrix is derived that embodies the epipolar geometry between a full perspective view and a weak perspective view. The systems and methods of the present invention preferably uses the derived fundamental matrix together with the 2D image information of the full and weak perspective views to digitally reconstruct the 3D object and produce results with multi-resolution processing techniques. These techniques include recovering and refining motion parameters and recovering and refining structure parameters of the fundamental matrix. The results can include, for example, 3D positions of points, camera position between different views, texture maps, and the like.

Other References

• Zhang Z. Determining the Epipolar Geometry and its Uncertainty: A Review. pp. 161-195 1997
• Xu G. and Zhang Z. Epipolar Geometry in Stereo, Motion and Object Recognition. Kluwer Academic Publishers. 1996
• Luong Q. and Faugeras O. The Fundamental Matrix: Theory, Algorithms, and Stability Analysis. International Journal of Computer Vision, 17, pp. 43-75, 1996
• Nash S. and Sofer A. Linear and Nonlinear Programming. 1996
• Shapiro L. Affine Analysis of Image Sequences. Cambridge University Press, pp. 1-210, 1995
• Hartley R. In Defence of the 8-point Algorithm. GE-Corporate Research and Development. pp. 1064-1070, 1995
• Huang T. and Netravali A. Motion and Structure from Feature Correspondences: A Review. Proceedings of the IEEE. vol. 82 No. 2, pp. 252-268, 1994
• Shashua A. Projective Structure from Uncalibrated Images: Structure frim Motion and Recognition. IEEE Trans. PAMI, vol. 16 No, 8, pp. 778-790, Aug. 1994
• Faugeras O. Three-Dimensional Computer Vision, The MIT Press, 1993
• Mohr R., Veillon F., and Quan L. Relative 3D Reconstruction Using Multiple Uncalibrated Images. pp. 543-548, 1993
• Haryley R., Gupta R., and Chang T. Stereo from Uncalibrated Cameras. GE-Corporate Research and Development. pp. 761-764, 1992
• Huang T.S. and Faugeras o. Some Properties of the E Matrix in Two-View Motion Estimation. IEEE Trans. PAMI, vol. 11, No. 12 pp. 1310-1312, Dec. 1989
• Aggarwal J.K. and Nandhakumar N. On the Computation of Motion from Sequences of Images--A Review. Proceedings Of the IEEE, vol. 75, pp. 917-935, No. 8, 1988
• Roach J. and Aggarwal J.K. Determining the Movement of Objects from a Sequence of Images. pp. 554-562, 1980
• Ullman S. The Interpretation of Visual Motion. The MIT Press. 1979
• Aloimonos J. Perspective approximations. vol. 8, No. 3 pp. 179-192
• Faugeras O. What can be seen in three dimensions with an uncalibrated stereo rig? pp. 563-578
• Golub G. and Van Loan C. Matrix Computations-Third Edition. The John Hopkins University Press
• Hartley R. Euclidean Reconstruction from Uncalibrated Views. pp. 237-257
• Maybank S. Theory of Reconstruction from Image Motion
• Spellucci P. A SQP Method for General Nonlinear Programs Using Equality Contrained Subproblems. Technical University at Darmstadt, Department of Mathematics. pp. 1-27