I guess since my company posted it on youtube, I can share what I've been working on.
We recorded 18 of these and we're a little past the halfway mark on post-production. The idea was to be able to do these in a super quick fashion, but the post-production has too many steps. Still, it's faster than actually animating the character and better than random mouth movements to indicate talking.
- Three of us sat in the studio with individual mics each with a separate camera on us. (two mics went into a camera, the 3rd was a usb mic). This way it had a more dynamic, live feel.
Premiere Pro part 1
- I took the footage into Premiere Pro, synced everything up in one timeline (I created a multi-cam sequence so auto-syncing was easy).
- The video and audio for each character was in separate tracks, so on each track I deleted all the audio from the other characters, so the audio track was silent for the other characters speaking.
- I then exported video clips for each of the three characters.
- Using the ManyCam app, I played the video clip for the close up of a character, and pressed keys to trigger certain expressions (well, I also set up a joystick with joy2key so I didn't use the keyboard to trigger things). The processes was 1) press record in Ch, press play in ManyCam, click back in Ch and position the pointer where I want the eyes to look when I press a L-mouse button). I then did some quick editing and pickups if I wanted to change a part of the performance.
- Export the video/audio
- Repeat the process to record that same character for the group shot
- Repeat steps 1-3 for the other two characters
- Sync up all the performances in the group scene (here's where having clapped before each show was very helpful)
- Export the video/audio of the group scene
Adobe Media Encoder or After Effects
Convert png sequences to mp4 files
Premiere Pro Part 2
- Import the close ups and group shots, select them and make them a multi-camera track
- Do a multi-cam edit, cutting out the flubs and messups
- Add the intro, and outro and ambient sound.
Things to make the process better/more productive (I think I've already made or seen feature requests for most of these) :
- The ability to processe pre-recorded video so I wouldn't need manycam or a webcam pointed at a monitor to get the motion.
- The ability to
- Audio waveform in the Ch timeline
- Faster png export - or - added mp4 (video) export - or - live link that removes the intermediary formats entirely
- A way to use the captured performance data from one puppet with a different puppet in a different scene. Right now I have to record each character twice, once for the close up and once for the wide shot. The video and audio are the same, just the puppet connections are different.
- A way to isolate audio tracks. It would be helpful to hear what the other characters were saying while recording one character, but I can't have the audio from the other characters in the video I'm recording from because then the mouths will be reacting to everyone. If I could monitor the left channel and record on the right channel, I could set that up when I export the clips in the first processing stage.
- keyboard shortcut for [Set Rest Post] so I wouldn't need to move the mouse if I need to reset the pose during a recording. (important when using mouse look)
- markers on tracks
The big things to make this quicker would be, not having to record 6 performances based on 3 videos, and not having to wait for pngs to export and then wait again for them to be turned into mp4s. Fortunately, I can use a different computer to handle that last bit.