6 Replies Latest reply on Aug 22, 2015 2:54 AM by sillybomb

    Smoothing out mouth animation

    nuttersanonymous! Level 1

      Hi, I haven't got the Character Animator yet and am just asking about it.

       

      From all the videos and tutorials I've seen so far, it looks like the movements of the eyes and body are fluid but the mouth seems to have only one frame per viseme - and this makes it look jittery, especially if the character is speeking slowly.

       

      Is it possible to smooth out the transition between the various viseme mouth shapes by either adding extra frames to help blend them, or to have the mouth move with a warp or liquify function (like it does with the program Auto lip-sync).

       

      Also, does the program's lip sync need the video footage to work fully or can it figure out the correct visemes to use from just an ordio file?

       

      Many Thanks

        • 1. Re: Smoothing out mouth animation
          CoSA_DaveS Adobe Employee

          Indeed, the Lip Sync behavior currently does mouth replacements in the style of common cartoon animation / stop-motion mouths. There is also the Nutcracker Jaw behavior that can make the mouth flap up & down smoothly, which you can choose to use instead of Lip Sync. In the future we might add smooth interpolation of mouth shape. You could try using AE's Pixel Motion frame interpolation as a post-process, but I don't think it would look very good.

           

          The lip sync can be generated from either a live performance or pre-recorded audio file.

          • 2. Re: Smoothing out mouth animation
            nuttersanonymous! Level 1

            I've roughly charted the transition frames needed between the 9 visemes - some seem to flow from one viseme to the next without needing any extra frames, some need 1 extra frame and a few need 2 extra frames inbetween. All these intermediate frames, I recon, can be copies of other visemes (eg. to go from "A,I" to "f,v" you can put "C,D,G,K,N" in between). So what you'd need is some kind of script or something that will insert the extra frames depending on which viseme follows which, taking the time scale into account.

             

            I have no idea about scripting or if there's any other way of doing this - do you think it would be an easy workaround or a nightmare?

            • 3. Re: Smoothing out mouth animation
              CoSA_DaveS Adobe Employee

              Interesting idea. This would be hard for you do, but possible for us with internal changes to the Lip Sync behavior. I'll add it to our backlog. If others are interested in this solution (introducing intermediate mouth shapes, using existing visemes, to smooth motion between visemes), please comment on this thread so we know about it.

              • 4. Re: Smoothing out mouth animation
                taylorg83855257

                I would love to see this!

                • 5. Re: Smoothing out mouth animation
                  dinowax

                  In Flash, I used to use motion clips for vowels and consonants. I understand there are cycles and my hope is to smooth it out with that feature. It would also be great to have a graph view for keyframes, like an automation window that you would use in Pro Tools. Love the technology happening here.

                  • 6. Re: Smoothing out mouth animation
                    sillybomb Level 1

                    Random thought for a workaround -- hide all layers except the mouth and perform a morph e.g. AE's Pixel Motion frame interpolation where required then motion track your result on to your animated body..