20 Replies Latest reply on Sep 18, 2008 12:21 PM by rafa@mediatech

    Impossible to Sync Audio with Lips

    DarrenDoesIt Level 1
      I understand that sound is measured in time and graphics in frames, but the sound shouldn't start from the beginning when the cursor is halfway into it. There must be a way around it.

      I'm animating frame by frame with no vectors. I put a .wav file in one of the sound channels and stretch it across the entire animation (few thousand frames) for a scene. Then, no matter where I am in the animation, when I press play the sound starts from the beginning.

      There is no way in hell I'll be able to sync lips with dialogue by playing the entire scene from the beginning every time just to see the sound synced with audio. I want to be able to play the same fraction of a second (several frames) while animating and KNOW WHERE IN THE SOUND I'M AT. I would have thought this expensive program would at least display the spectrogram or something, much less not play the audio from the beginning when I'm in the middle of the sound sprite. I've prerecorded the entire sound score in one Audacity file and exported it as a .wav to Director. I don't want to use any other sound channels or ques, etc.

      PLEASE HELP, and thank you.

        • 1. Impossible to Sync Audio with Lips
          James Newton, ACP Level 3
          A sound sprite in the score is like other animated sprite, such as filmLoops or animated GIFs: it will start playing when the playback head jumps to a frame where it exists.

          If you need more control over where it starts, you will need to take a look at the Lingo controls for Sound Channels: Sound Channel on livedocs

          You can find a scrubber for sound here.
          • 2. Re: Impossible to Sync Audio with Lips
            DarrenDoesIt Level 1
            Thank you, that was actually the answer I was looking for, makes sense when I think of it that way (it acts like any other sprite). I'll see if I can get into some lingo -- it'll take a bit of work but not as much trouble as introducing another program into the equation. Looked into it before a bit but wasn't sure if that was the right track so I'll look further into it, and thanks again.

            --Darren
            • 3. Impossible to Sync Audio with Lips
              rafa@mediatech Level 1
              Hi Darren:
              I came to the same realization of the audio issue (not an issue, just the way they decided to make it in Director). The great thing about the expensive program you bought is that there are always more than one ways of achieving something.

              I find that adding cue points to my audio helps a lot when I need to sync my sprite members.

              I would suggest to break your long audio piece into a few audio pieces. Add cue points to your audio. When you import the files to Director, it will recognize these cue points and this will give you more control over your audio member. Now, instead of expanding the audio member to a few thousand frames, you will be able to control with a few hundred!!! (maybe even less!)

              Dean has an awesome tutorial on how to control the audio clips, take a look at it, and it will help you a lot.

              http://www.deansdirectortutorials.com/

              Look at tutorial 6 and 7...this should get you on your way.

              good luck!

              Rafael.
              • 4. Re: Impossible to Sync Audio with Lips
                DarrenDoesIt Level 1
                Copy that Rafael,

                Thanks for the input. I'll go ahead and and acquiesce into the cue point idea, shouldn't be too big a problem. Guess I just got frustrated with the fact that there's not really a single all-inclusive program for what I need but rather an array of gigantic programs each washed with features I don't need. What happened to some of the more "to the point" straightforward single-task-focused programs of the DOS days?

                --Darren
                • 5. Re: Impossible to Sync Audio with Lips
                  Level 7

                  "DarrenDoesIt" <webforumsuser@macromedia.com> wrote in message
                  news:garnhf$kbn$1@forums.macromedia.com...
                  > Copy that Rafael,
                  >
                  > Thanks for the input. I'll go ahead and and acquiesce into the cue
                  > point
                  > idea, shouldn't be too big a problem. Guess I just got frustrated
                  > with the
                  > fact that there's not really a single all-inclusive program for what
                  > I need but
                  > rather an array of gigantic programs each washed with features I
                  > don't need.
                  > What happened to some of the more "to the point" straightforward
                  > single-task-focused programs of the DOS days?
                  >
                  > --Darren


                  Hi darren,
                  if you can explain what you try to do in detail, maybe I or someone
                  else here can help you find what you need.
                  You must give some more hints though, now its just guessing.

                  What did you try? What is not working? What do you have?

                  Did you check the links I sent you before?

                  Good luck,
                  Richard.


                  • 6. Impossible to Sync Audio with Lips
                    rafa@mediatech Level 1
                    Darren:
                    I understand your frustration. I do not think there is a software out there that does it all! (somebody should make it). If you think about it, you can have after effects...it is great for making effects on your videos, but it is not recommended to use for your FINAL version of the video. For this you need something like Final Cut, or Premiere or Sony Vegas...And then, if you want to make a DVD or Blu-ray, you would need something like DVD Architect or DVD Encore.
                    I think all these software are just tools (like a carpenter would use many tools to build something).

                    Adobe should make the "Swiss army knife" of software...one that does it all.

                    Like Richard said, if you explain what you are trying to accomplish with more detail, we might be able to point you in the right direction.

                    Rafael.
                    • 7. Re: Impossible to Sync Audio with Lips
                      DarrenDoesIt Level 1
                      Okay,

                      Since everyone's been kind enough to stick with me I'll try to explain what I'm doing, but its a bit of a ramble.

                      I'm combining techniques of traditional animation with cutout animation. I've created simple character designs on paper. I then draw and color body parts on paper and create photoshop files of each body part. Using the wand selection tool I relatively quickly cut the drawing and make it a sort of "digital cutout" by creating a photoshop the drawing I cut over a blank background layer. For example if I want, say, a head to turn, I organized several of these photoshop files (each a tween of the head turning) into a folder and labeled the folder as a sequence for that particular action (HEAD_TURN_LEFT). With this method I've created walk cycles, heads squashing/stretching/turning/shaking, etc.

                      I asked a buddy of mine who was in charge of a computer animation department at a local university, "Say I want a piece of software that will act more as a cut-out studio than a vector animation program. I want to be able to switch the body out every time it turns or switch out several arms frame by frame for when I want the character to raise its hand (instead of rotating a single arm upward each frame). This is my nightmare hybrid of cutout / cel techniques."

                      Basically he replied that I should look into director so I decided to take my work over to someone who owns the program. No I haven't made the mistake of blowing money on Director yet because I'm only working with a test scene. I'm in the process of testing director for my needs.

                      I was ecstatic to find that the sprite/cast member structure of director truly seemed to suit my needs perfectly and, yes, too perfectly. I rendered several .avi files and found that the final product was what I was going for, which is a sort of early Mike Judge look.

                      I decided that I could animate each shot of my scene as a separate file in Director and edit them together with an editing program. I chose to import a background and three characters to the program and then add sound, preparing to animate to the sound. This is where I finally ran into trouble and realized that I was foolish....

                      I created the sound file in Audacity by mixing some 16 or 17 tracks into a stereo .wav file that I intended to import into Director into one of the sound channels. When I imported the sound it quickly became clear that this program not only won't bring in the spectrogram, but also how the program merely knows to cue sounds at specific frames but has nothing to do with synching an entire single file.

                      I am now clear about the fact that the program is more oriented for playing small amounts of footage, not scenes. That would be fine because each file is just a short shot, but I'm not intending to put sound to animation. Rather, I'm intending to put animation to sound. The most tedious task will be lip sync, else there'd be a way around it. So I've dead-ended. I mean, isn't there at least a way to divide the sound into MANY single frame sprites? THAT would solve everything.

                      Whew. Let me know if that makes any sense at all, and if not I understand. The main point here is that even if I should have to choose another piece of software, its going to need to use the same sprite/member concept. Editing software won't solve the problem and especially not after effects. One program is good for one thing, and another is good for another, so if I could just put an end to the issue by managing to stick with Director or a program very much like it, that would be preferable. Thank you for falling victim to this blabber.

                      Sincerely,

                      Darren
                      • 8. Re: Impossible to Sync Audio with Lips
                        DarrenDoesIt Level 1
                        I should add that lip syncing is tough to do without the spectrogram or at least without the ability to work with accurate sync sound playback.

                        --Thanks
                        • 9. Impossible to Sync Audio with Lips
                          wgb14 Level 1
                          One simple suggestion. Why don't you try software tools that are designed specifically to do what you want (e.g., Crazytalk) and then import the result into director for further manipulation? Crazytalk and other similar programs offer automatic lip sync, they are very cheap (in the range of $50-$100) and you can perform this kind of operations without woring about anythign as the software takes care of the process for you!!!
                          • 10. Re: Impossible to Sync Audio with Lips
                            DarrenDoesIt Level 1
                            I'd prefer if someone who is not a spammer would answer that question, but thanks. I've looked into lipsync software and I'll only use it if its free, maybe. But not Crazytalk, thank you.
                            • 11. Impossible to Sync Audio with Lips
                              wgb14 Level 1
                              <<I'd prefer if someone who is not a spammer would answer that question>>

                              A spammer? What do you mean? I am not trying to advertise anything here, all I am saying is that you should look into other software that is spefically designed to accomplish what you want, simply becuase Director is not. Read my post carefully please, it says for EXAMPLE Crazytalk. I am sure that there are several others out there free or with very low cost. There is no point cursing ADOBE for making you buy other software, as Director can not cover the special needs of everybody.

                              BTW Please look at the number of posts before you call someone a spammer again. Thank you
                              • 12. Re: Impossible to Sync Audio with Lips
                                DarrenDoesIt Level 1
                                okay well whether or not you're a spammer I still am positive I don't want that kind of software for the job, but thank you and I apologize.
                                • 13. Re: Impossible to Sync Audio with Lips
                                  rafa@mediatech Level 1
                                  Hey Darren:
                                  I don't think my next comment would fall in the "spammer" category, but here goes...

                                  I think in one of your other posts you said you have After Effects. I believe AE is the perfect tool for what you are trying to do. It allows you to parent/child your pieces (which by the way, you can import your PSD directly into AE). It will show the audio wave (like you want). And I now the latest version has great animation tools (like the one that actually rigs layers that you can animate). I would make all my animations in AE, and then use Director or Flash to show the movie files (if you need to make your own player thing). If not...I think you could just use AE and be done with it...

                                  Rafael.
                                  • 14. Re: Impossible to Sync Audio with Lips
                                    Production Monkey Level 3
                                    "...its going to need to use the same sprite/member concept. Editing software won't solve the problem and especially not after effects..."

                                    What do you have against AE? Have you ever tried it? Because you seem to think that it is a vector animation tool which is simply wrong.

                                    It does have the same "sprite/member" concept as Director as well as a layered timeline where you can make frame accurate changes to individual frames. You can import your image sequences as individual objects/members and drag those onto the timeline. You can change the frame rate. You can scrub the audio or start the playback anywhere in the timeline and the audio will playback at the correct time index. You can see the waveform and apply effects. You can animate any effect or property you desire. How can this not be the tool you want to use?

                                    You can try to force Director to do what you want with cue points, code, and what ever; but I feel you are just setting yourself up for a lot of hurt.

                                    Lastly, "South Park" is created in After Effects. That style of animation sounds a lot like what you are trying to do.

                                    Good luck.
                                    • 15. Re: Impossible to Sync Audio with Lips
                                      DarrenDoesIt Level 1
                                      Okay Raf / Production Monkey / Everybody,

                                      It seems as though you are adamantly insisting upon After Effects so I'll give it another shot. But one more thing: should I import everything as cropped layers?

                                      I need to know how to set the right interpolation (hold interpolation?) and I still don't see anything resembling the structure of Director, which comes up immediately when you open Director but not when you open After Effects. After Effects seems to insist on using vectors and automatic interpolation, not to say its not capable of doing what I want. Any tips on setting the stage? Perhaps I can leave you alone after that! Thanks for the help.
                                      • 16. Re: Impossible to Sync Audio with Lips
                                        DarrenDoesIt Level 1
                                        P.S. The most important problem I experienced with after effects was that it was a real pain to work with frame by frame -- I must have been doing something wrong. Not only could I not find an interface similar to Director but it also ALWAYS applies changes and manipulations to ALL frames, which is worthless to me. Just let me know why I'm stupid.
                                        • 17. Re: Impossible to Sync Audio with Lips
                                          DarrenDoesIt Level 1
                                          Also, I've heard both that South Park uses After Effects and that they use Maya. In after effects everything seems to revolve around the stopwatch feature. Well say I want to keep switching different objects out in the same postition to create the illusion of life/movement? Last I tried, every time I would cut an object it would cut it from every frame. Same with any manipulation. I could not find an interpolation that would instruct AE to apply these changes only to the current frame.
                                          • 18. Re: Impossible to Sync Audio with Lips
                                            rafa@mediatech Level 1
                                            Hmmm.
                                            You do not need to crop the layers in photoshop to import them into AE. AE will bring them as layers, and you can manipulate them individually.

                                            I don't understand what you mean when you say:
                                            "every time I would cut an object it would cut it from every frame"
                                            Are you cutting the entire layer??? You can duplicate the layer and have it appear in another time of you time line...you could also use key frames and just change the opacity (from 0-100% and 100-0%), this would make it look like it is going away...

                                            What do you mean by:
                                            "on using vectors and automatic interpolation"

                                            After effects uses whatever you bring in, images, video, audio...anything.

                                            If you are trying to apply things to a single frame, just duplicate the layer, extend it for the amount of time you want it to last, apply the changes. Once the time you desire passes, have the original layer show up on the time line.

                                            Rafael.
                                            • 19. Re: Impossible to Sync Audio with Lips
                                              DarrenDoesIt Level 1
                                              Rafael,

                                              Okay, I have no problem with duplicating the layer. The problem is that say ten thousand frames later I'm going to have ten thousand duplicated layers to plow through. This still isn't like the sprite/member concept because it would be pretty utterly confusing to work with down the line. With the sprite/members I could create a sequence that would be displayed horizontally with different sprites, copy the sequence, paste it several times, and then I would have for instance someone strumming a guitar. I'm not sure how AE's design would be comfortable to work with several thousand layers and frames down the road. How can I keep that organized? How can I copy and paste the same sequence of movements to create i.e. a walk cycle or something.
                                              • 20. Re: Impossible to Sync Audio with Lips
                                                rafa@mediatech Level 1
                                                I think I understand better what you are trying to do...
                                                I used a program called Flipbook to create animations when i did my internship. We (the interns) would receive the stack of paper from the animators (each single drawing with its frame number info). Flipbook would scan the image and add it to the next frame. When I was done scanning, I would hit play and VOILA! animation!

                                                I know this might sound reaaaally hard to do in AE, but if you have digital files, I think it is the way to go. You can create separate animations with however many layers, then comp all this layers, and add it to your main comp. Creating "compositions" of your layers is a good way to organize in AE.

                                                You can search for other animation software that might help you. I think it can be achieve on AE...it is just a matter of you trying to work with what you have and making it work.

                                                good luck!
                                                Rafael.