Translingual visual speech synthesis

Patent 6813607 Issued on November 2, 2004. Estimated Expiration Date:

January 31, 2020. Estimated Expiration Date is calculated based on simple USPTO term provisions. It does not account for terminal disclaimers, term adjustments, failure to pay maintenance fees, or other factors which might affect the term of a patent.

Abstract Claims Description Full Text

Patent References

Sound-synchronized video system
Patent #: 5608839
Issued on: 03/04/1997
Inventor: Chen

Method and apparatus for producing audio-visual synthetic speech
Patent #: 5657426
Issued on: 08/12/1997
Inventor: Waters, et al.

Method and apparatus for synthetic speech in facial animation
Patent #: 5878396
Issued on: 03/02/1999
Inventor: Henton

Method for generating photo-realistic animated characters
Patent #: 5995119
Issued on: 11/30/1999
Inventor: Cosatto, et al.

Coarticulation method for audio-visual text-to-speech synthesis
Patent #: 6112177
Issued on: 08/29/2000
Inventor: Cosatto, et al.

Method and apparatus for diphone aliasing
Patent #: 6122616
Issued on: 09/19/2000
Inventor: Henton

Talking facial display method and apparatus
Patent #: 6250928
Issued on: 06/26/2001
Inventor: Poggio, et al.

6317716

Speech driven lip synthesis using viseme based hidden markov models
Patent #: 6366885
Issued on: 04/02/2002
Inventor: Basu, et al.

Face synthesis system and methodology
Patent #: 6449595
Issued on: 09/10/2002
Inventor: Arslan, et al.

More ...

Inventors

Application

No. 09494582 filed on 01/31/2000

US Classes:

704/276, Pattern display704/258, Synthesis704/260, Image to speech704/270Application

Examiners

Primary: To, Doris H.
Assistant: Opsasnick, Michael N.

Attorney, Agent or Firm

Foreign Patent References

0674315 DE 09/01/1995
05-298346 JP 11/01/1993
WO9946732 WO 09/01/1999

International Classes

G10L 1100
G11B 27032

Abstract

A computer implemented method in a language independent system generates audio-driven facial animation given the speech recognition system for just one language. The method is based on the recognition that once alignment is generated, the mapping and the animation hardly have any language dependency in them. Translingual visual speech synthesis can be achieved if the first step of alignment generation can be made speech independent. Given a speech recognition system for a base language, the method synthesizes video with speech of any novel language as the input.

Other References

R.E. Donovan, et al., “The IBM Trainable Speech Synthesis System”, International Conference on Speech and Language Processing, 1998.
E.D. Petajan, et al., “An Improved Automatic Lipreading System to Enhance Speech Recognition”, Proc. OHI,1988, pp. 19-25.
T. Chen et al., “Audio-Visual Integration in Multimodal Communication”, Proceedings of the IEEE, vol. 86, No. 5, May 1998.
F. Lavagetto, et al., “Lipreadable Frame Animation Driven by Speech Parameters”, 1994 International Symposium on Speech, Image Processing and Neural Networks, Apr. 13-16, 1994, Hong Kong.