8 Replies Latest reply on Jul 18, 2016 12:53 PM by claytor1999

    Manually Changing Visemes

    mksalva Level 1

      I have been trying Character Animator for the past few days. I am mostly interested in using it for lip sync, but when I load audio files, I'm not getting as many mouth movements as I would like.

       

      I created characters based on the puppets that come with the program. (ie - Ah, D, Ee, etc.) I set the frame rate to 30 FPS.

       

      The audio was recorded in a professional studio, and the dialogue is easily understood by the human ear, but I'm seeing less than half of mouth movements in the animations, compared to what I how I would do this manually.

       

      Is there a way to "Compute Lip Sync From Scene Audio" and then add the additional mouth images that CA glaringly passed over?

       

      Thanks!

        • 1. Re: Manually Changing Visemes
          CoSA_DaveS Adobe Employee

          You can select the Lip Sync Audio Input take bar and choose Timeline > Split Lip Sync into Visemes. Then each viseme will get its own take bar that you can slide, trim, or delete.

           

          Unfortunately we haven't finished this editing UI, so it's still fairly primitive. Notably, with each take bar in its own track, they cascade up the screen inconveniently. They are unlabeled. And to create new ones you have to enable the Lip Sync Keyboard Input in the properties panel (when the puppet is selected) and then use keys A/D/E/F/L/M/O/R/S/U/W to trigger the chosen mouth and choose Timeline > Record 1-frame Take (or 2-frame). Zero triggers the neutral mouth.

           

          So, yeah, still work in progress. Right now it's mainly useful for lengthening or shortening visemes.

           

          I'd recommend lowering your scene frame rate to 24fps instead of 30fps, at least if you want a more traditional look to the animation.

           

          We're still actively working to improve the lip sync. If you're willing to send us a portion of your audio file, we can look at the results to see where it's not working. We can then use that to make the mouth selection more accurate. Even better if you can provide your hand-tweaked result that you'd consider perfect.

          2 people found this helpful
          • 2. Re: Manually Changing Visemes
            CoSA_DaveS Adobe Employee

            Oh, and make sure you're using Preview 4 -- its lip-sync is significantly better than Preview 3. It also has more mouth shapes.

            1 person found this helpful
            • 3. Re: Manually Changing Visemes
              mksalva Level 1

              Thank you for your reply.

               

              I was not using Preview 4. I had put the version 3 shortcut in my dock, and I'm not sure if I knew 4 was available. (I have 4 now.)

               

              Do I manually change the visemes by holding down the A/D/E/F/L/M/O/R/S/U/W keys, and at the same time, hold down Command-1 or Command-2? If I hold down the letter key first, the mouth movement goes away as soon as I take my finger off the key.

               

              I will be glad to supply you with a sample of the audio I am using. Let me know how you'd want to do that.

               

              I think it's great that Adobe is making this available, if only because I've been doing lip-sync cartoon animation for years in AE. A great deal of my time is spent on the lip-sync alone, and this would make life so much easier. (Currently, I have been making a Quicktime file with all the mouth pictures, and using time-remapping to manually keyframe all the mouth pics.)

              • 4. Re: Manually Changing Visemes
                Jim Acquavella Adobe Employee

                I just tried this on windows and it works.  Try this:

                 

                1.  Enable Keyboard Input for the Lip Sync Behavior.

                2.  Hold down a key that invokes a viseme.

                3.  Press Command+1

                4.  Then release all keys.

                 

                You should see a new take in the timeline for the Lip Sync behavior.  Play through it (or move the time needle over the take).  Does the viseme appear? 

                1 person found this helpful
                • 5. Re: Manually Changing Visemes
                  Jim Acquavella Adobe Employee

                  If that doesn't work, you can hold the key down and use the menu commands that do the same thing. See the timeline menu.

                  • 6. Re: Manually Changing Visemes
                    claytor1999

                    Thanks Jim... I couldn't figure out the mechanics of making this working. Appreciated.

                    • 7. Re: Manually Changing Visemes
                      Jim Acquavella Adobe Employee

                      Robb, what specifically doesn't work?  You should be able to enable the Keyboard Input parameter from the Lip Sync behavior, then use the keyboard to activate a specific viseme, then use the menu command (Timeline->Record 1-Frame Take) or (Timeline->Record 2-Frame Take) to create a take that represents that viseme.  If that doesn't work, walk me through the steps and let me know where you get stuck. 

                      • 8. Re: Manually Changing Visemes
                        claytor1999 Level 1

                        Sorry Jim... my reply meant that your explanation helped me to get it to work. Not that it wasn't working. It was end user error on my part. Thanks.