Methods and devices for producing and using synthetic visual speech based on natural coarticulation

Patent 6539354 Issued on March 25, 2003. Estimated Expiration Date:

March 24, 2020. Estimated Expiration Date is calculated based on simple USPTO term provisions. It does not account for terminal disclaimers, term adjustments, failure to pay maintenance fees, or other factors which might affect the term of a patent.

Abstract Claims Description Full Text

Patent References

3463885

3585303

Apparatus for providing facial image animation
Patent #: 4104625
Issued on: 08/01/1978
Inventor: Bristow , et al.

Creating visual images of lip movements
Patent #: 4260229
Issued on: 04/07/1981
Inventor: Bloomstein

Apparatus for non-invasive measurement and display nasalization in human speech
Patent #: 4335276
Issued on: 06/15/1982
Inventor: Bull , et al.

Cassette player and slide assembly
Patent #: 4835556
Issued on: 05/30/1989
Inventor: Spector

Speech synchronized animation
Patent #: 4884972
Issued on: 12/05/1989
Inventor: Gasper

Computer card
Patent #: 4951203
Issued on: 08/21/1990
Inventor: Halamka

Erasable animated drawing board
Patent #: 4976647
Issued on: 12/11/1990
Inventor: Axtell

Animated holographic stereogram display
Patent #: 5191449
Issued on: 03/02/1993
Inventor: Newswanger

More ...

Inventors

Assignee

Fluent Speech Technologies, Inc.

Application

No. 535100 filed on 03/24/2000

US Classes:

704/260, Image to speech345/423, Tessellation704/235, Speech to image704/258Synthesis

Examiners

Primary: Banks-Harold, Marsha D.
Assistant: Abebe, Daniel

Attorney, Agent or Firm

Marger Johnson & McCollom P.C.

International Class

G10L 013/08

Abstract

A method of producing synthetic visual speech according to this invention includes receiving an input containing speech information. One or more visemes that correspond to the speech input are then identified. Next, the weights of those visemes are calculated using a coarticulation engine including viseme deformability information. Finally, a synthetic visual speech output is produced based on the visemes' weights over time (or tracks). The synthetic visual speech output is combined with a synchronized audio output corresponding to the input to produce a multimedia output containing a 3D lipsyncing animation.

Other References

Luo "a novel approach for classifying speech into mouth shapes . . . " 1994 IEEE pp 465-468.
Zhong "acoustic driven viseme identification for face animation" 1997 IEEE pp 7-12.
Olives "towards a high quality finish talking head" IEEE 1999 pp 433-437.
Goff "a text to audiovisual speech synhtesizer for french" IEEE pp 2163-2166.
Kshirsagar "prinicipal components of expressive speech animation" 2001 IEEE pp 38-44.
Chen "visual speech synthesis using quadtree splines" IEEE 2001 pp 153-156.
Ezzat "mike talk" "a talking facial display based on morphing visemes" IEEE.
Yang "automatic selection of visemes for image-based visual speech synthesis" 2000 IEEE pp 1081-108