Face synthesis system and methodology

Patent 6449595 Issued on September 10, 2002. Estimated Expiration Date:

March 11, 2019. Estimated Expiration Date is calculated based on simple USPTO term provisions. It does not account for terminal disclaimers, term adjustments, failure to pay maintenance fees, or other factors which might affect the term of a patent.

Abstract Claims Description Full Text

Patent References

TV Movies that talk back
Patent #: 4569026
Issued on: 02/04/1986
Inventor: Best

Speech synchronized animation
Patent #: 4884972
Issued on: 12/05/1989
Inventor: Gasper

Fast search method for vector quantizer communication and pattern recognition systems
Patent #: 4907276
Issued on: 03/06/1990
Inventor: Aldersberg

Sound-synchronized video system
Patent #: 5608839
Issued on: 03/04/1997
Inventor: Chen

Advanced tools for speech synchronized animation
Patent #: 5630017
Issued on: 05/13/1997
Inventor: Gasper, et al.

Method and apparatus for producing audio-visual synthetic speech
Patent #: 5657426
Issued on: 08/12/1997
Inventor: Waters, et al.

Device and method for dubbing an audio-visual presentation which generates synthesized speech and corresponding facial movements
Patent #: 5826234
Issued on: 10/20/1998
Inventor: Lyberg

Method and apparatus for synthetic speech in facial animation
Patent #: 5878396
Issued on: 03/02/1999
Inventor: Henton

Automated synchronization of video image sequences to new soundtracks
Patent #: 5880788
Issued on: 03/09/1999
Inventor: Bregler

Automated speech alignment for image synthesis
Patent #: 5884267
Issued on: 03/16/1999
Inventor: Goldenthal, et al.

More ...

Inventors

Assignee

Microsoft Corporation

Application

No. 266011 filed on 03/11/1999

US Classes:

704/235, Speech to image348/515, Audio to video434/118, COMPUTER LOGIC, OPERATION, OR PROGRAMMING INSTRUCTION434/169, Electrical component included in teaching means434/185, Speech704/238, Distance704/249, Subportions704/258Synthesis

Examiners

Primary: Chawan, Vijay

Attorney, Agent or Firm

Foreign Patent References

0 689 362 EP. 12/13/1995
0 710 929 EP. 05/13/1996
2 231 246 GB. 11/13/1990
WO 97/36288 WO. 10/13/1997

International Class

G10L 015/26

Abstract

A system and method for synthesizing a facial image, compares a speech frame from an incoming speech signal with acoustic features stored within visually similar entries in an audio-visual codebook to produce a set of weights. The audio-visual codebook also stores visual features corresponding to the acoustic features. A composite visual feature is generated as a weighted sum of the corresponding visual features, from which the facial image is synthesized. The audio-visual codebook may include multiple samples of the acoustic and visual features for each entry, which corresponds to a sequence of one or more phonemes.

Other References

Chou et al., ("Speech recognition for image animation and coding", 1995, International Conference on Acoustics, Speech, and Signal Processing, 1995, ICASSP-95, May 1995, vol. 4, pp. 2253-2256).
Williams et al., ("Frame rate and viseme analysis for multimedia applications", IEEEE First Workshop on Multimedia Signal processing, Jun. 1997, pp. 23-25).
Gao et al., ("Synthesis of facial images with lip motion from several real views", Third International Conference on Automatic Face and Gesture Recognition Proceedings, Apr. 1998, pp. 181-186).
Ostermann (Animation of synthetic faces in MPEG-4, Proceedings Computer Animation 98, Jun. 1998, pp. 49-55).
McAllister et al., ("Automated lip-sync animation as a telecommunications aid for the hearing impaired", Proceedings 1998 IEEE 4th Workshop Interactive Voice Technology for telecommunications Applications, Sep. 1998, pp. 112-117).
Olives et al., ("Towards a high quality Finnish talking head", 1999 IEEE 3rd Workshop on Multimedia Signal Processing, Sep. 1999, pp. 433-437).
Ezzat et al., ("Miketalk: a talking facial display based on morphing visemes", Proceedings Computer Animation 98, Jun. 1998, pp. 96-102).
Yang et al., ("Automatic selection of visemes for image-based visual speech synthesis", 2000 IEEE International Conference on Multimedia and Expo, 2000, Aug. 2000, vol. 2, pp. 1081-1084).
Faruquie et al., ("large vocabulary audio-visual speech recognition using active shape models", Proceedings, 15th international Conference on Pattern recognition, Sep. 2000, pp. 106-109).
Sirovich, L. et al., "Low-Dimensional Procedure for the Characterization of Human Faces," Journal of the Optical Society of America, vol. 4, No. 3, pp. 519-524 (Mar. 1, 1987