5 Replies Latest reply on Mar 24, 2016 5:12 PM by CoSA_DaveS

    Three puppet Character Animator show example

    teamicm Level 1

      Hi,

      I guess since my company posted it on youtube, I can share what I've been working on.

       

       

      We recorded 18 of these and we're a little past the halfway mark on post-production. The idea was to be able to do these in a super quick fashion, but the post-production has too many steps. Still, it's faster than actually animating the character and better than random mouth movements to indicate talking.

       

      Recording

      • Three of us sat in the studio with individual mics each with a separate camera on us. (two mics went into a camera, the 3rd was a usb mic). This way it had a more dynamic, live feel.

       

      Premiere Pro part 1

      1. I took the footage into Premiere Pro, synced everything up in one timeline (I created a multi-cam sequence so auto-syncing was easy).
      2. The video and audio for each character was in separate tracks, so on each track I deleted all the audio from the other characters, so the audio track was silent for the other characters speaking.
      3. I then exported video clips for each of the three characters.


      Character Animator

      1. Using the ManyCam app, I played the video clip for the close up of a character, and pressed keys to trigger certain expressions (well, I also set up a joystick with joy2key so I didn't use the keyboard to trigger things). The processes was 1) press record in Ch, press play in ManyCam, click back in Ch and position the pointer where I want the eyes to look when I press a L-mouse button). I then did some quick editing and pickups if I wanted to change a part of the performance.
      2. Export the video/audio
      3. Repeat the process to record that same character for the group shot
      4. Repeat steps 1-3 for the other two characters
      5. Sync up all the performances in the group scene (here's where having clapped before each show was very helpful)
      6. Export the video/audio of the group scene

       

      Adobe Media Encoder or After Effects

      Convert png sequences to mp4 files

       

      Premiere Pro Part 2

      1. Import the close ups and group shots, select them and make them a multi-camera track
      2. Do a multi-cam edit, cutting out the flubs and messups
      3. Add the intro, and outro and ambient sound.
      4. Export.

       

      Things to make the process better/more productive (I think I've already made or seen feature requests for most of these) :

      • The ability to processe pre-recorded video so I wouldn't need manycam or a webcam pointed at a monitor to get the motion.
      • The ability to
      • Audio waveform in the Ch timeline
      • Faster png export - or - added mp4 (video) export - or - live link that removes the intermediary formats entirely
      • A way to use the captured performance data from one puppet with a different puppet in a different scene. Right now I have to record each character twice, once for the close up and once for the wide shot. The video and audio are the same, just the puppet connections are different.
      • A way to isolate audio tracks. It would be helpful to hear what the other characters were saying while recording one character, but I can't have the audio from the other characters in the video I'm recording from because then the mouths will be reacting to everyone. If I could monitor the left channel and record on the right channel, I could set that up when I export the clips in the first processing stage.
      • keyboard shortcut for [Set Rest Post] so I wouldn't need to move the mouse if I need to reset the pose during a recording. (important when using mouse look)
      • markers on tracks

      The big things to make this quicker would be, not having to record 6 performances based on 3 videos, and not having to wait for pngs to export and then wait again for them to be turned into mp4s. Fortunately, I can use a different computer to handle that last bit.

        • 1. Re: Three puppet Character Animator show example
          oksamurai Employee Moderator

          This is great feedback - thank you for the detailed explanation of your process and one of the final results! You'll be happy to know several of the things you've listed will be improved in the next release.

           

          One thing I've wanted is a behavior you could add to get instant natural motion. Like imagine the sway of the trees if you added a pivot/sway behavior and could adjust the intensity, randomness, etc of it. Currently, one way to make the process faster would be to record 30 seconds of data, then duplicate it, drag it to the right, and blend the intro so it looks like it seamlessly cuts into the previous take.

           

          For the different shots - one technique I've used is to record each puppet individually w/o a background at a high res, and then composite them together into After Effects (see https://www.youtube.com/watch?v=aS5Qg1o6Kqs). That way, you're only recording once for each character. You could also record at a closeup level of fidelity for the whole scene, and then manually reposition / scale for different shots in PR or AE.

           

          Thanks again!

          • 2. Re: Three puppet Character Animator show example
            teamicm Level 1

            I also like the idea of behaviors to control swaying.  I animated something in AE and made cycle layers in the Photoshop file to handle that.

             

            I was trying to avoid dealing with video with an alpha channel since it's so much more cumbersome in Premiere since the video formats that support it are few and it's a bit more taxing on the computer... or at least it used to be. Also, I had the multicamera edit in mind when I developed the workflow and was trying to avoid using After Effects for simplicity's sake. (I'm proficient in AE, so it wasn't an intimidation issue ). Now that you mention it, if this computer could handle a resolution high enough to get the close up AND the full body shot, I could just record once, and use AE to frame the shot variations and export the four movie clips from there. I may try it. I don't know if it'll save a tremendous amount of time at this point, since I'll have to alter the puppets, but it could save a few hours for the last 5 episodes. I won't have to double up the recording, wait for the group shot to export, sync the three clips up in Ch which is a little more difficult than in AE. Plus, the animation will definitely match from the close up to the wide shot. The downside is that if it's in AE, I may be tempted to refine the animation or add more effects to it.

             

            Thanks for the input.

            • 3. Re: Three puppet Character Animator show example
              CoSA_DaveS Adobe Employee

              Yes, great feedback, thank you! A few specific comments:

               

              > The ability to processe pre-recorded video so I wouldn't need manycam or a webcam pointed at a monitor to get the motion.

               

              See this thread for the AE workflow for this: Re: animation using recorded video

               

              > added mp4 (video) export - or - live link that removes the intermediary formats entirely

               

              The former we expect to be in Preview 4, the latter later


              > A way to isolate audio tracks. It would be helpful to hear what the other characters were saying while recording one character, but I can't have the audio from the other characters in the video I'm recording from because then the mouths will be reacting to everyone.


              The workaround for this is to use headphones, so you can hear the existing audio, but the microphone can't.


              > keyboard shortcut for [Set Rest Post]


              Cmd-P (mac) / Ctrl-P (win)

              • 4. Re: Three puppet Character Animator show example
                teamicm Level 1

                Thanks for the reply!

                 

                After doing a test episode the big takeaway was that the export time for the higher resolution of the full characters (since the characters are trees - they are large) completely negates any savings in having to capture a performance more than once.

                 

                Can we get more export options? png sequences are nice and convenient, but they take a lot of processing power to compress.

                 

                For example. I did a test AE scene. 1920 x 1080 15 seconds long. Solid with animated turbulent noise and a mask on it. All exported with transparence or alpha channel

                Exported as PNG sequence: 1 Min 38 Sec, 1 Min 40 Sec, 1 Min 39 Sec

                Exported as Targa sequence: 5 Sec, 6 Sec

                Exported as TIFF sequence: 9 Sec

                Exported as TIFF sequence (LZW compression on): 19 Sec

                Exported as OpenEXR (piz compression): 38 Sec (not that we need the color depth)


                I imagine there is a similar hit in reading PNG sequences since the computer has to decompress the images.

                • 5. Re: Three puppet Character Animator show example
                  CoSA_DaveS Adobe Employee

                  Good point. If you'd like early access to Preview 4, message me directly with your AdobeID email address. (BTW, decompress is usually way faster than compress.)