System and method for expression mapping

Patent 6828972 Issued on December 7, 2004. Estimated Expiration Date:

April 24, 2022. Estimated Expiration Date is calculated based on simple USPTO term provisions. It does not account for terminal disclaimers, term adjustments, failure to pay maintenance fees, or other factors which might affect the term of a patent.

Abstract Claims Description Full Text

Patent References

Method and apparatus for producing an image of a person's face at a different age
Patent #: 4276570
Issued on: 06/30/1981
Inventor: Burson , et al.

Advanced tools for speech synchronized animation
Patent #: 5689618
Issued on: 11/18/1997
Inventor: Gasper, et al.

Facial image processing method and facial image processing apparatus
Patent #: 5850463
Issued on: 12/15/1998
Inventor: Horii

Digital cartoon and animation process Patent #: 6393134
Issued on: 05/21/2002
Inventor: Tostevin, et al.

Inventors

Assignee

Microsoft Corp.

Application

No. 10131972 filed on 04/24/2002

US Classes:

345/473Animation

Examiners

Primary: Nguyen, Phu K.

Attorney, Agent or Firm

International Class

G06T 1500

Abstract

A system and method for mapping facial expressions. Facial expressions exhibit not only facial feature motions, but also subtle changes in illumination and appearance (e.g., facial creases and wrinkles). These details are important visual cues, but they are difficult to synthesize. Traditional expression mapping techniques consider feature motions while the details in illumination changes are ignored. A novel technique for facial expression mapping is presented. The illumination change of one person's expression is captured in what is called expression ratio image (ERI). Together with geometric warping, an ERI is mapped to any other person's face image to generate more expressive facial expressions.

Other References

T. Beier and S. Neely. Feature-based image metamorphosis. In Computer Graphics, pp. 35-42. Siggraph, Jul. 1992.
Boissieux, G. Kiss, N.m. Thalman and P. Karla. Simulation of Skin Aging and Wrinkles with Cosmetic Insights. In Computer Animation and Simulation 2000,2000.
J. Blinn, Models of light reflection for computer synthesized pictures. In Computer Graphics, pp. 192-198, SIG-GRAPH, Jul. 1977.
C. Choi, Age Change for Predicting Future Faces. In IEEE International Fuzzy Systems Conference Proceedings, pp. 1603-1608, 1999.
P. E. Debevec. Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In Computer Graphics, Annual Conference Series, pp. 189-198, 1998.
P. E. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker,W. Sarokin, and M. Sagar. Acquiring the reflectance field of a human face. In Computer Graphics, Annual Conference Series, pp. 145-156. Siggraph, Jul. 2000.
R. Epstein, A. Yuille, and P. Belhumeur. Learning object representations from lighting variations. In ECCV 96 International Workshop, pp. 179-199, 1996.
G. Healey and T. Binford. Local shape from specularity. Computer Vision Graphics and Image Processing, pp. 62-86, Apr. 1988.
A. Lanitis, C. Taylor, and T.Cootes. Modeling the process of aging in face images. In IEEE Proceedings of the 7th International Conference on Computer Vision, pp. 131-136, Sep. 1999.
P. Litwinowicz and L. Williams. Animating images with drawings. In Computer Graphics, pp. 235-242. Siggraph, Aug. 1990.
Z. Liu, Y. Shan, and Z. Zhang. Expressive expression mapping with ratio images. In Computer Graphics, Annual Conference Series, pp. 271-276. Siggraph, Aug. 2001.
S. R. Marschner and D. P. Greenberg. Inverse lighting for photography. In IST/SID Fifth Color Imaging Conference, Nov. 1997.
S. R. Marschner, B. Guenter, and S. Raghupathy. Modeling and rendering for realistic facial animation. In Rendering Techniques, pp. 231-242. Springer Wien NewYork, 2000.
T. Riklin-Raviv and A. Shashua. The quotient image: Class based rerendering and recognition with varying illuminations. In IEEE Conference on Computer Vision and Pattern Recognition, pp. 566-571, Jun. 1999.
H. Rushmeier, G. Taubin, and A. Gueziec. Applying shape from lighting variation to bump map capture. In Eurographics Workshop on Rendering, pp. 35-44, 1997.
Y. Wu, P. Kalra, and N.M. Thalmann. Physically-based wrinkle simulation & skin rendering. In Eurographics Workshop on Computer Animation and Simulation, pp. 69-79, 1997.