5 Replies Latest reply on May 2, 2008 4:04 PM by Rothrock

    Programs that will synch character's lips to a voice track?

    Level 7
      I was told by someone that there are programs that will automatically
      synch lips and eye motion to a vocal track. Does anyone know anything
      about these?

      Thanks.
        • 1. Re: Programs that will synch character's lips to a voice track?
          ooba Level 1
          Depends:
          Are you talking about a service that you can buy if so,
          there are a few programs (services) out there that you can pay for that can take an audio file and synch it to lips. As for the eyes, I dont see how an audio track would do that. These services out there can generate for you an animated swf file for your site but as far as I can tell you do NOT get the program. You get access to the service they provide off of their servers where u submit the audio file and maybe even a text file and it will return to you the final product.

          If you are refering to something that you can leverage to help you create your own lip synching audio to mouth or text to speeach flash engine / animation.

          I have been working on for the past month just such a thing with another developer who does .NET. He used the microsoft text to speech API where we pass it in text and it generates for me an xml file that specifies the timestamp and phoneme to use at a given point in time of the audio file created.

          When I am playing the audio file i compare the current playhead position to the times given in the xml to determine what to display to the user.

          Working out well so far still doing upgrades to it.

          If you want names of things out there Site Pal is one of them.

          • 2. Re: Programs that will synch character's lips to a voice track?
            Rothrock Level 5
            I would be interested in knowing as well. I know that there is a company called CodeBaby that offers something like this. It generates swf files, but it isn't interactive in real time and it isn't Flash.

            For Flash AS2 I would think it would have to be some kind of external program that parsed the audio and generated an XML or some other kind of list. Since in AS2 there is not really any access to the audio data. AS3 has access to the data and could probably do some kind of real time analysis.

            Other than that I would love to find out what anybody else knows....
            • 3. Re: Programs that will synch character's lips to a voice   track?
              Level 7
              I am referring to a 3rd party external program (I think external and not
              a plugin for Flash).

              I guess it is possible he was referring to a service and not a program.

              Thanks.

              ooba wrote:
              > Depends:
              > Are you talking about a service that you can buy if so,
              > there are a few programs (services) out there that you can pay for that can
              > take an audio file and synch it to lips. As for the eyes, I dont see how an
              > audio track would do that. These services out there can generate for you an
              > animated swf file for your site but as far as I can tell you do NOT get the
              > program. You get access to the service they provide off of their servers where
              > u submit the audio file and maybe even a text file and it will return to you
              > the final product.
              >
              > If you are refering to something that you can leverage to help you create your
              > own lip synching audio to mouth or text to speeach flash engine / animation.
              >
              > I have been working on for the past month just such a thing with another
              > developer who does .NET. He used the microsoft text to speech API where we pass
              > it in text and it generates for me an xml file that specifies the timestamp and
              > phoneme to use at a given point in time of the audio file created.
              >
              > When I am playing the audio file i compare the current playhead position to
              > the times given in the xml to determine what to display to the user.
              >
              > Working out well so far still doing upgrades to it.
              >
              > If you want names of things out there Site Pal is one of them.
              >
              >
              >
              • 4. Programs that will synch character's lips to a voice track?
                Walter Elias Level 2
                I've used Toon Boom Studio for this. (www.toonboom.com). It's an excellent animation program for Flash output, and has the lip sync function built in. However, I must say the lip sync is not very sophisticated. It works best if you're using a clearly-enunciated midwestern American accent. Try anything else and you get bizarre results. What with the state of voice recognition technology as it currently stands, it's hard to imagine a lip sync program that does a whole lot better than that at this point.
                • 5. Re: Programs that will synch character's lips to a voice track?
                  Rothrock Level 5
                  I just found out that Poser 7 has a Talk Designer function. I haven't tried it yet, but I think I will.