10 Replies Latest reply on May 23, 2018 11:29 PM by CoSA_DaveS

    Smoothing out mouth animation

    nuttersanonymous! Level 1

      Hi, I haven't got the Character Animator yet and am just asking about it.

       

      From all the videos and tutorials I've seen so far, it looks like the movements of the eyes and body are fluid but the mouth seems to have only one frame per viseme - and this makes it look jittery, especially if the character is speeking slowly.

       

      Is it possible to smooth out the transition between the various viseme mouth shapes by either adding extra frames to help blend them, or to have the mouth move with a warp or liquify function (like it does with the program Auto lip-sync).

       

      Also, does the program's lip sync need the video footage to work fully or can it figure out the correct visemes to use from just an ordio file?

       

      Many Thanks

        • 1. Re: Smoothing out mouth animation
          CoSA_DaveS Adobe Employee

          Indeed, the Lip Sync behavior currently does mouth replacements in the style of common cartoon animation / stop-motion mouths. There is also the Nutcracker Jaw behavior that can make the mouth flap up & down smoothly, which you can choose to use instead of Lip Sync. In the future we might add smooth interpolation of mouth shape. You could try using AE's Pixel Motion frame interpolation as a post-process, but I don't think it would look very good.

           

          The lip sync can be generated from either a live performance or pre-recorded audio file.

          • 2. Re: Smoothing out mouth animation
            nuttersanonymous! Level 1

            I've roughly charted the transition frames needed between the 9 visemes - some seem to flow from one viseme to the next without needing any extra frames, some need 1 extra frame and a few need 2 extra frames inbetween. All these intermediate frames, I recon, can be copies of other visemes (eg. to go from "A,I" to "f,v" you can put "C,D,G,K,N" in between). So what you'd need is some kind of script or something that will insert the extra frames depending on which viseme follows which, taking the time scale into account.

             

            I have no idea about scripting or if there's any other way of doing this - do you think it would be an easy workaround or a nightmare?

            • 3. Re: Smoothing out mouth animation
              CoSA_DaveS Adobe Employee

              Interesting idea. This would be hard for you do, but possible for us with internal changes to the Lip Sync behavior. I'll add it to our backlog. If others are interested in this solution (introducing intermediate mouth shapes, using existing visemes, to smooth motion between visemes), please comment on this thread so we know about it.

              • 4. Re: Smoothing out mouth animation
                taylorg83855257

                I would love to see this!

                • 5. Re: Smoothing out mouth animation
                  dinowax

                  In Flash, I used to use motion clips for vowels and consonants. I understand there are cycles and my hope is to smooth it out with that feature. It would also be great to have a graph view for keyframes, like an automation window that you would use in Pro Tools. Love the technology happening here.

                  • 6. Re: Smoothing out mouth animation
                    sillybomb Level 1

                    Random thought for a workaround -- hide all layers except the mouth and perform a morph e.g. AE's Pixel Motion frame interpolation where required then motion track your result on to your animated body..

                    • 7. Re: Smoothing out mouth animation
                      arthurh48697603

                      HI - Has there has been any updates on this issue between then and now? My wife and I both felt like a smooth morph between mouth viseme shapes would take things to the next level. Thanks.. love the program!

                      • 8. Re: Smoothing out mouth animation
                        alank99101739 Level 4

                        I have always assumed you could just use a Cycle Layers behavior under the Vizeme name, but realized I never got around to trying it. That is, under “Ah” instead of providing a graphic, make it a sublayer and insert the different stages of the animation, then put a “Cycle Layers” behavior on the ”Ah” group (with appropriate properties).

                         

                        I assume CH is just triggering different sublayers under the Mouth group, so I would expect cycle layers to work (just like they do with other triggers).  I just have not tried it myself (and am not near a computer just now...)

                        • 9. Re: Smoothing out mouth animation
                          arthurh48697603 Level 1

                          Yes, it would seem this is the best approach at this time. For our particular project, we have many characters and for each frame in our story they are different, or unique in their poses and expressions. So, we were looking to save time with a "button" to push to smooth transitions between viseme. My wife found that creating the vowel sounds work better than the consonants. We're new at this!   Thanks for the quick response.

                          • 10. Re: Smoothing out mouth animation
                            CoSA_DaveS Adobe Employee

                            When assigning Cycle Layers behavior to visemes, you probably want to use the Hold on Last Layer option (so the mouth doesn't disappear on sounds that held for a long time), and On Trigger End = Stop immediately (so that a new viseme starting isn't delayed by a previously unfinished cycle).

                            1 person found this helpful