1 Reply Latest reply on Jan 11, 2016 11:46 AM by oksamurai

    Lip-sync 2.0?

    Fred Sprinkle

      First of all, I'm really excited about the work the Character Animator team is doing.


      So, high-five to everyone involved.


      Concern: I don't feel the audio file based lip-sync technology is at a professional level.


      Explanation: Even when using a clean vocal-only track I find the lip-sync is still a bit weird. Maybe the algorithm is off, maybe the mouth shape choices need to be revisited, but it always looks a bit...uncanny. A more user-friendly phoneme editor might help, but I think something needs some tweaking at a base level. It's like the algorithm is not sure what mouth shapes to make "the hero" in each word or phrase. I think in manual lip-syncing humans decide what mouth shape is the hero in each word, but  maybe the algorithm has to guess. It's hard to put into words.


      Having said this, I can't even imagine how difficult it would be to get a computer to figure out what sounds a person has made, so no hard feelings.


      Thanks for listening, not sure I can explain much beyond this.