4 Replies Latest reply on Jun 29, 2012 11:22 AM by Pierre Devereux

    Lip Sync Plug in

    Pierre Devereux Level 2



      Has anyone had any good experiences with a specific Lip-Sync tool or Plug in?


      I have created a set of PSD files with different mouth positions, showing various Phonemes. I have so far, only played a bit with Magpie - but I believe it is no longer supported or available for purchase. Our characters are still, so the mouth movements must be added in After Effects. I also imported each phoneme PSD file into after effects, layered them one per frame and sequenced them one after the other. I then add them to a background image using time remap. It works pretty well, but is rather time consuiming, and if it has to be done for a few hours of footage, well.... I know nothing comes easy, but that seems a bit rough to me.


      Magpie had a pretty interresting export to After effects feature, that allowes the user to export the keyframe settings, and add that to the time remapped layer. Unfortunately, as mentioned, I think Magpie is no longer available.




      Any suggestions from the forum for a good program, or plug in?


      Thanks guys,


      Have a great weekend



        • 1. Re: Lip Sync Plug in
          Rick Gerard Adobe Community Professional & MVP

          There's no completely automatic way of doing this, but there are several techniques. Aharon Rabinowitz has a great tutorial using time remapping to animate mouth shapes. You can also use creative scaling or other deformation techniques by shooting your voice over talent's face while they read. Simply put some tracking marks just above the upper lip, just below the lower lip, and in the corners of the mouth, then stabilize the shot, track the stabilized shot, and then tie the movements of the tracking marks to scale or deformations using other options.

          • 2. Re: Lip Sync Plug in
            Mylenium Most Valuable Participant

            Depends what exactly you want. If it needs to represent exact phonemes, time remapping is your only option. If it can be deformation based, there is quite a few ways to attach effects to tracking data. I once did a custom setup for a client using Freeform Pro and that would also work with e.g. Bezier Warp to an extent. It's basically based on stuff that is used in Symmetrion and PointGlue:XYZ from my site combined and refined...



            • 3. Re: Lip Sync Plug in
              Pierre Devereux Level 2

              Thanks guys,


              What I am looking for, is to put some sort of mouth (I say - "some sort", as I am still proofing either a simple black stripe, warped and distorted to actual mouth images drawn and coloured to match the characters) on our characters and have them speak, not 100% realistic, but to give them a little life. The characters are created from pewter, so movement is not possible on them.


              I have seen the tutorial from CreativeCow for the time remapping and it looks like this will be the easiest option. I have done a bit of work with the distortion tools in the past, and it also looks pretty good mind you.....


              I found a pretty cool program called Magpie, that allows you to create a set of mouths, then by assigning and tweaking settings, the program can take a section of sound, and automatically assign a mouth image to that section. This would make it nice and easy then, after creating a bunch of mouths to quickly get a list of key-frames to attach to your time remapping effect. The problem is, that Magpie does not seem to be available any more! Pity - are any of you aware of any programs that replace it?


              Other than that, I will try some of the mouth tracking suggestions today - hmmm, hours of my own mouth on film - what fun! :-) - and see what sort of effects I can get out of that.


              Thanks for the suggestions, and thanks for the links - Ill try it all out and report back on what I found to be the best way forward (just for anyone who is interrested *even if that person exists only in my own head!* )





              • 4. Re: Lip Sync Plug in
                Pierre Devereux Level 2



                just a quick update:


                I have done a few tests with the Phenome Time remap Lip-Sync, and the outcome is pretty good - if not a little "Choppy". I also tried this in conjunction with Re:Flex morph, but the effect is not what i was hoping for.


                I have a question - has anyone used the program DAZ3D? It has a built in lip sync that seems to be pretty darn good! - now to try and figure out how to export with it, and maybe how to get some different skins for the lips. I have only just taken a very quick look at it, so lets see what happens next week.


                Thanks guys,