10 Replies Latest reply on Apr 4, 2014 8:39 AM by Dallas Kruse

    Easiest way to lip sync cartoons in AE

    Dallas Kruse

      I have drawn some cartoons up in Illustrator and i want them to "sing along" with the lyrics of a song in this music video I'm working on.

       

      here's how I'm guessing one does it....

       

      1. create and draw the characters in illustrator

      2. create different mouth positions in different layers for the vowels and consonents (Ah, F, V, L, etc)

      3. Import that Illustrator file into AE and add all the layers into the timeline

      4. "mute" the vowels and consonents i don't need and "unmute" them when i need them for a word.

       

      My question is ...

      I'm doing a short 3 second clip and I'm animating flowers lip syncing "La La La La".

       

      My "L" mouth position happens 16 times in those 3 seconds when singing the repeated phrase "La La La La".  The only way I've figured out how to have them repeat is to Duplicate the "L Mouth Position" layer 16 times and then drag and trim the footage of that mouth position into the correct place.  This seems superfluous to me and I'm sure there's another way to do it. 

       

      I thought about the opacity keyframes but that basically just fades it in and out.  Is there an "on/off" keyframe or effect that I can automate INSTEAD of having to have 16 layers of one mouth position?

       

      If I did this with each vowel/consonant and each character, I'd have probably over a hundred layers! 

       

      Any way to expedite what I'm trying to do?

        • 2. Re: Easiest way to lip sync cartoons in AE
          Dallas Kruse Level 1

          Great resources.  I'll check these out.

          • 3. Re: Easiest way to lip sync cartoons in AE
            bogiesan Level 4

            Dallas Kruse wrote

            here's how I'm guessing one does it....

            Character animation is a long-established, perfected craft. We've been doing it for more than 100 years.

            In addition to Todd's links to resources that can help you learn how to do phonemes in AE, head to your local library and check out everything they have on the subject of film animation.

             

            A fundamental in animation is to only expend as much effort as necessary to communicate an illusion of movement or life. IN the olden days of film exposed at 24 frames per second, animators (except those who were working at Disney) discovered they only needed to draw 12 unique frames per second, known sometimes as "shooting on twos," and the results were good enough to sell most illusions. Lip sync was reduced to 6 to 12 basic mouth shapes that changed infrequently; too many mouthshapes looked choppy while too few was a lack of speech resolution that was difficult to discern.

             

            In the early days of AE, we used Motion Math to analyze audio levels and apply the distortion to the mouth image layer. Now we can use parenting to lock a mouth precomp to a character's face and expressions to instantly switch out phonemes with only a few keyframes.

            • 4. Re: Easiest way to lip sync cartoons in AE
              Dallas Kruse Level 1

              bogiesan wrote:

               

              Dallas Kruse wrote

              here's how I'm guessing one does it....

              Character animation is a long-established, perfected craft. We've been doing it for more than 100 years.

              In addition to Todd's links to resources that can help you learn how to do phonemes in AE, head to your local library and check out everything they have on the subject of film animation.

               

              A fundamental in animation is to only expend as much effort as necessary to communicate an illusion of movement or life. IN the olden days of film exposed at 24 frames per second, animators (except those who were working at Disney) discovered they only needed to draw 12 unique frames per second, known sometimes as "shooting on twos," and the results were good enough to sell most illusions. Lip sync was reduced to 6 to 12 basic mouth shapes that changed infrequently; too many mouthshapes looked choppy while too few was a lack of speech resolution that was difficult to discern.

               

              In the early days of AE, we used Motion Math to analyze audio levels and apply the distortion to the mouth image layer. Now we can use parenting to lock a mouth precomp to a character's face and expressions to instantly switch out phonemes with only a few keyframes.

              I do realize that animation is an artform that is older than I've been alive ... and I respect the people who do it for a living.

               

              WIth that being said ... I'm trying to add to my palette of abilities and this exploration is not only challenging, but fun as well.

               

              In my Comp, I have a Precomp of 3 mouth positions (smiling, "L", and "Ah") for the lip syncing for about 4 seconds of footage.

              I have puppetted the flower to "dance" along with the beat of the music and simply copied the pin points to the mouth layers.

              I precomped those mouth layers and put them in the timeline (as instructed in the timeline). 

              The problem is, when i add the precomp to the comp, the puppet movements aren't working correctly.  Some of the lines and part of the graphic is stays in one position while half of the mouth moves along with the puppet pin points.

               

              I couldn't figure out what I was doing wrong.  I could use keyframes and set which frame i wanted to use in correlation to the mouth position if the flower stayed in one position...but since it was "dancing", I couldn't get the tutorials advice to work.

               

              What I did was just duplicate each mouth position as many times as that mouth position was needed (about 13 times) and trimmed the layer and manually placed them in the correct position.  Once all the manual positioning and editing was done (took about 45 minutes), i precomped those layers and that seemed to work.  Seems like the long way of doing it but I figured it out.

               

              NOW, the problem is, when I RAM preview, everything looks fine.  But when I Render out, one of the mouth positions is off.  It's not off in the RAM preview nor in the file that is outputted when I "SAVE RAM PREVIEW".  But, when I Render it, the mouth position gets thrown off.  I'm not sure how this could happen...any ideas?

              • 5. Re: Easiest way to lip sync cartoons in AE
                Todd_Kopriva Level 8

                > " NOW, the problem is, when I RAM preview, everything looks fine.  But when I Render out, one of the mouth positions is off.  It's not off in the RAM preview nor in the file that is outputted when I "SAVE RAM PREVIEW".  But, when I Render it, the mouth position gets thrown off.  I'm not sure how this could happen...any ideas?"

                 

                 

                 

                Show screenshots, especially of your Composition panel.

                • 6. Re: Easiest way to lip sync cartoons in AE
                  Rick Gerard Adobe Community Professional & MVP

                  What are your render settings?

                  • 7. Re: Easiest way to lip sync cartoons in AE
                    bogiesan Level 4

                    Dallas Kruse wrote:

                    I do realize that animation is an artform that is older than I've been alive ... and I respect the people who do it for a living.

                    ...

                    What I did was just duplicate each mouth position as many times as that mouth position was needed (about 13 times) and trimmed the layer and manually placed them in the correct position.  Once all the manual positioning and editing was done (took about 45 minutes), i precomped those layers and that seemed to work.  Seems like the long way of doing it but I figured it out.

                    ....

                    NOW, the problem is, when I RAM preview, everything looks fine.  But when I Render out, one of the mouth positions is off.  It's not off in the RAM preview nor in the file that is outputted when I "SAVE RAM PREVIEW".  But, when I Render it, the mouth position gets thrown off.  I'm not sure how this could happen...any ideas?

                    1. I wasn't denegrating, just lecturing. 

                    2. Duplication of the mouth parts is not necessary. When you have more experience with AE in general, you will understand how layers can be swithed on and off as needed without duplication.

                    3. You'll also know about parenting. You want to precomp your mouth parts (call the comp PHONEMES) and then parent Phonemes to the face of your puppet. Whither goeth the puppet, there, too, goeth Phonemes.

                    4. The differences between RAM previews and regular previews? Sorry, I cannot imagine what that issue might be caused by.

                    • 8. Re: Easiest way to lip sync cartoons in AE
                      Dallas Kruse Level 1

                      1. appreciated

                      2. Now that I know that layers can be turned on and off, i'll research how to do so.

                      3. I have been learning about precomping but not yet dealt with parenting much.  I will research that as well.

                      4. I'm sure it's user error ... I'm only about 3 days into AE.

                       

                      RE: render settings ... I just have them set at BEST SETTINGS/QUALITY right now as I'm unfamiliar with the ins and outs of render settings.  usually it gives me a MASSIVE file and then I'll throw it in COMPRESSOR and slim it down for dropbox and viewing on my iPhone or remote laptop.

                      • 9. Re: Easiest way to lip sync cartoons in AE
                        Szalam Adobe Community Professional & MVP

                        If you're only three days into AE, you should really take a step back and go through these resources first. This will give you a solid foundation to build on. Otherwise you'll end up frustrated.

                         

                        Your workflow of rendering a lossless file and then compressing it for final delivery is the recommended workflow. Most folks tend to use a codec that gives a somewhat smaller file size though (Avid's DNxHD codec is quite popular for this as is Cineform - both of these are free and, while not technically lossless, are close enough for what you're doing).

                        • 10. Re: Easiest way to lip sync cartoons in AE
                          Dallas Kruse Level 1

                          Szalam wrote:

                           

                          If you're only three days into AE, you should really take a step back and go through these resources first. This will give you a solid foundation to build on. Otherwise you'll end up frustrated.

                           

                          Your workflow of rendering a lossless file and then compressing it for final delivery is the recommended workflow. Most folks tend to use a codec that gives a somewhat smaller file size though (Avid's DNxHD codec is quite popular for this as is Cineform - both of these are free and, while not technically lossless, are close enough for what you're doing).

                          Well, for clarification ... the first 2 days were going through start up tutorials and such.  I'm moving "fairly" well for the things I need to do and appreciate ALL the superior help thus far.