1 2 Previous Next 51 Replies Latest reply on Feb 15, 2018 12:35 PM by matthewa75473542 Go to original post
      • 40. Re: How to edit video to audio with a sub-frame precision?
        dkitsov Level 2


        The '2 Pip' is a single frame of audio.  It has been used for decades as the SMPTE Standard reference for synch in the countdown leader.


        The Frame of audio and the single frame of video are aligned and there is no need for subframe adjustment.


        Heres the fact: 


        You can not consciously see less than one frame of video at 24 frames (1/48 sec) or more per second. Thats what the film industry was developed around.


        BTW: Post a link to your video so we can judge what the issue is.

        There is no such thing as a single frame of audio, there is a single samples. What you are referring to is a multitude of audio samples limited in their duration by the duration of a system with a lower sample rate - ie. video frame.


        24 frames per second is not about the quality of what you see -  it was accepted as a standard by the industry because it is the slowest frame rate that allowed for somewhat acceptable level of audio synch.

        Films befora that were shot and displayed at lower framerates without being percievesd as a collection of still frames by the audience.

        Here is the fact: you might not consciously see a 1/24 frame of action and I cannot use certain display divices as even I 60/th of a second not high enough frame rate for me not to see individual frames - people are different.

        • 41. Re: How to edit video to audio with a sub-frame precision?
          dkitsov Level 2

          shooternz wrote:


          They are such devices, just for different applications. Look it up.


          I am keen to expand my understanding and  to look up what you refer to,  but I do not know what the "seach term" would be.   Point me in the direction please.







          Without the synch generator certain things are impossible in a current film production ie. when shooting 3d with two cameras they must be in a perfect synch. Once action is called both of the cameras must start shooting at the same time. If first camera starts shooting and the second starts shooting 1/48 or 1/96 or 1/128 of second after the first they are not in synch - two images that are produced and seem to be the same temporally - are not: there will be difference between two frames from both devices that are not caused by parallax but by bad synch - is noticeable to human - causes headaches.

          So just like this synch generator with a genlock there must be technology that can lock together both a recording device (camera) and a playback device (original music track) for purposes of accurate synch during recording of a music video.

          • 42. Re: How to edit video to audio with a sub-frame precision?
            the_wine_snob Level 9

            So just like this synch generator with a genlock there must be technology that can lock together both a recording device (camera) and a playback device (original music track) for purposes of accurate synch during recording of a music video.

            With cine, this locking of sync between the film and audio was easy. Depending on the equipment used, one either used the recorder to generate the sync, or used the camera to generate. The two devices were tethered, so the sync signal could be sent from the Master device to the Slave device.


            Now, the "clocks" in current equipment is much more accurate than the simple crystals in the older cine gear, so the general assumption is that that accuracy will be 100% between all devices. Unfortunately, it is often NOT, as different mfgrs. use different clocks. Today, we ofen do not have any cables between devices, and one is relying on 100%. I feel that the technology exists for 100% accuracy, but it seems that each mfgr. is just doing their "thing," and assuming that it will be "good enough." In many cases, it is, but then in others, it is not. As an example, many of the Zoom recorders require that their output be adjusted slightly for sync with many cameras. With the accuracy of the clocks, one can usually test to find out how much adjustment needs to be made, and then just make a note of that - applying that adjustment to ALL output from that device.


            It should be easy to get 100% accuracy between all pro, and even pro-sumer gear, but it seems to be a sandlot argument, "It's HIS fault," as the audio mfgr. points at the camera mfgr. "No, it's HIS fault," as the finger swings the other way. We ought to lock them both in a room with some cold beers, and not let them out, until they agree to adopt the same exact clock for their gear...



            1 person found this helpful
            • 43. Re: How to edit video to audio with a sub-frame precision?
              dkitsov Level 2

              I agree with what you say. However, for the purposes of the original problem that I posted, as far as production rather than the postproduction is concerned, one wouldn't even need absolute accuracy, as long as one gets repeatability. I do not need the playback of original soundtrack to start at the same moment that the first line and the first column of the video is being read from the sensor - it can be late, as long as the latency stays the same. For example: if all of my audio playback for the onset audio reference starts 0.2 seconds later than the camera is rolling - well, that is beautiful - I will be easily able to adjust this delay because all of the clips will have similar delay.

              As far as the original question of how to deal with it in post production I now have two different solutions and I will post them later.

              • 44. Re: How to edit video to audio with a sub-frame precision?
                Jim_Simon Level 9
                "It's HIS fault," as the audio mfgr. points at the camera mfgr.


                It's definitely the audio guys.  Seems that all DV cameras have no issues syncing with each other.  So the camera guys got it right.  It's only the audio devices that can't seem to get it right.  They sync neither with the cameras, nor each other.


                No cold beers for them.

                • 45. Re: How to edit video to audio with a sub-frame precision?
                  the_wine_snob Level 9

                  If you use the same equipment (and there are no issues with either), you should be able to get that repeatability.


                  Jim Simon, who is in this thread, has posted of the disparity between his cameras, and his separate audio recorders. He just tested, until he had the % of error between the video and the audio, and applies that correction. I do not recall his having any later issues with that workflow, but if so, I feel that he will make mention of them.


                  Good luck, and hope that you find that magic correction %, and that it always holds true, so you can get the repeatability.



                  • 46. Re: How to edit video to audio with a sub-frame precision?
                    toootone Level 1

                    dkitsov  I feel your pain!  I am an audio guy (trained sound engineer - recording studio and live sound), singer and musician, and I deal with these issues all the time.  Many video guys understand a little of this, but I have not run into a single video guy (main expertise is camera operation or video editing) who understands all of the issues of achieving perfect audio / video sync for a music video production.


                    There are at least 3 issues at play here in regards to synchronization:

                    1. Singer / Musician Performance
                    2. Audio playback word clock and Camera video sync / genlock
                    3. 1 camera or many?  1 take or multiple takes?


                    Because of the limitations and outright impossibility of sub-frame audio editing in most video editing software I do all of my audio for video in a dedicated audio DAW.  I use Logic Pro which is a fantastic deal in the Apple App Store for $200.  You get more software and plugins now than ever before, and it used to cost $1000.  So, many of my comments below are specifically about my use of Logic pro, but can be applied to pretty much any DAW.


                    Singer / Musician Performance


                    Music video production is best done by performing to pre-recorded audio, so that the performance is consistent between multiple takes.  The performers can usually be perfectly with the music playback most of the time, but there will inevitably be moments when they are not.  How do I fix this?  I used to take the vocal track from the multitrack recording and slice it up into hundreds of regions, shift them around, and crossfade them together.

                    Now, I use Melodyne almost exclusively (I promise - this is not a Melodyne commercial - I just love the software).  With Melodyne, I can shift the timing of any word in a vocal track, I can even shift mouth sounds within a word to match the video.  Imagine the singer sings the word "amazing"  In the video, she holds out the "m" longer, but in your audio multitrack, the "m" is sung shorter.  Melodyne, can extend the "m" within the word to match the video.


                    What if you don't have a multitrack studio recording, but just a stereo mix?  Melodyne can even de-mix the recording so that you can work on just the vocal, then put it back together.  Watch it done here Melodyne: Demixing and remixing vocals - YouTube.


                    Another tool I have used is Revoice Pro - This takes a studio recording (nicer sounding)  and will automatically match timing to a live recording (maybe not so nice sounding).  This really only works if you have an isolated track of both the studio vocal and the live vocal.  So, I always record multitrack audio when filming a music video.


                    Audio playback word clock and Camera video sync / genlock


                    The BEST setup for maintaining perfect sync between audio and video is to use pro-grade equipment that can sync together perfectly.  Anyone who used do live video feed switching with analog equipment is familiar with video sync or genlock.  See here Black and burst - Wikipedia, the free encyclopedia and here Genlock - Wikipedia, the free encyclopedia .  If your video wasn't in sub-frame sync, you would get very bad artifacts when you switched from one video signal to another.  To achieve perfect frame sync between video cameras and between video and audio, all devices need to be genlocked - video cameras, audio playback, audio recording.  Your cameras have to be able to accept timecode and genlock connections and your audio hardware has to be able to sync to timecode and audio word clock.  Then, you need a device that will connect everything together in perfect sync such as this ESE DV-321 Genlockable HD/SD Sync Generator or this Mutec MC-32 SD/HD Video & Digital Audio Sync Master Clock Generator .  With this kind of a setup, all of your devices will be in PERFECT frame-accurate sync.


                    Here is a very good article that explains it quite a bit better than I.  It also explains that yes, you can just shoot it and pray, assuming that you can align everything in software, but that you can still end up up to a half-frame off.  And if you are filming say, an hour show,  You could end up more than a frame off between devices if you don't use genlocked sync.

                    Timecode versus Sync: How They Differ and Why it Matters | explora


                    Another great article:

                    http://www.xdcam-user.com/2014/09/the-right-way-to-get-good-timecode-sync-with-multiple-ca meras/


                    1 camera or many?  1 take or multiple takes?


                    If you have a low budget, it is common to shoot something like a music video with just one camera, running the song many times to get all the angles and shots.  Low budget shoots don't have audio or video equipment that will sync together.  Sometimes this works well, sometimes not quite.


                    Here's a shoot I did with one camera, and audio playback from a phone through a bluetooth speaker:

                    The Duttons - $1.7 Million Titanic Violin - Nearer My God To Thee on Vimeo

                    It actually turned out quite nicely.


                    Here's another one - The Duttons Christmas on PBS - Sneak Peek - YouTube  - a higher budget multi-camera shoot, but audio and video were not synced together with genlock accuracy.  you'll notice that some clips are in perfect sync, but others are slightly out of sync - as little as 1/2 or 1/4 frame (a couple of places a performer is singing a word out of sync).  You CAN see it if you watch closely.  dkitsov is absolutely right, if all the clips were 1/2 frame off, you might not notice, but since some are in sync perfectly, and some aren't, you notice.


                    The chance of having more video clips that are some fraction of a frame out-of-sync is much greater if you are jam syncing multiple cameras, or you record multiple takes.


                    Final Thoughts


                    Here's my process to achieve best sync possible:

                    1. Cut video as best you can in video editor
                    2. Export reference video and audio tracks
                    3. load audio and reference video into DAW
                    4. Load any external audio such as studio mix or multitracks
                    5. for best sync comparison, use your EARS not your EYES
                    6. Solo 2 audio tracks that you want in sync, pan one left and one right, for example live sound from camera on left, studio recording on right
                    7. Nudge one by frames, then half frames, then even smaller - I use 1/8 frames
                    8. When you get close, the audio will start to phase (sound spacey)
                    9. If you get in sync, the phasing will diminish or stop and they will sound perfectly together
                    10. If you have some clips that are in sync and some that aren't, you will need to come up with a happy medium
                    11. Shift the audio 1/8 frame or 1/4 frame and see if it looks good for both clips
                    12. Now export audio from the exact beginning of the reference video
                    13. Import audio back into video editor to see your sync improvements.


                    ... Hope this was helpful, and good luck!

                    • 47. Re: How to edit video to audio with a sub-frame precision?
                      toootone Level 1

                      the_wine_snob wrote:



                      Now, the "clocks" in current equipment is much more accurate than the simple crystals in the older cine gear, so the general assumption is that that accuracy will be 100% between all devices.

                      That has never been the assumption.  It's true that today's gear is usually better at staying in sync when you just freewheel it, but the assumption has ALWAYS been that the oscillating crystal that provides the clock frequency for each device will never be precise enough to achieve perfect genlocked sync between devices.  That is why genlock syncing exists.  In the audio world it's called word clock.  Most often when connecting digital audio inputs and outputs wordclock sync is achieved through the same connection used to transmit the digital audio.  Any pro or even many semi-pro audio device has a BNC word clock connector on it.  This can be connected to other audio gear or to a video / audio genlock sync device to sync together perfectly.  I used to use the MOTU Digital Timepiece for this purpose.


                      Also, you really need both genlocked sync and timecode sync, otherwise you run into the very real and common problem of audio being up to 1/2 frame off from take to take even if the gear is running at exactly the same speed.

                      • 48. Re: How to edit video to audio with a sub-frame precision?
                        howiemnet Level 1

                        I have the same basic problem (lack of sub-frame positioning) but for different reasons: I'm creating a corporate animation in time with a track of library music.


                        While I'm blocking it all out in a comp, figuring out the pacing of the messages I've got to show, I often find I need to make a quick edit to the audio - eg chop out 8 bars of the tune so the animation will finish in time with the final beats - but it's pot luck whether the edit will work perfectly or whether you'll notice the timing jitter (even with a cross-fade).


                        Completely depends on the audio track and where you're cutting - sometimes an audio edit is good enough to use, if the beats happen to fall right on the frames.


                        Of course audio edits ought to be done in an audio package, but it's AE I'm using to try and nudge my content and my audio into some semblance of shape/pace/flow.


                        So sub-frame positioning of audio would definitely have value for me, if only to make blocking quicker and save me roundtripping the audio to Logic just for a quick edit

                        • 49. Re: How to edit video to audio with a sub-frame precision?
                          andrew_articom_sthlm Level 1

                          Hi everybody,


                          I also believe in sub-frame syncing, and I have a workaround that I've used for the occasional sync problem.


                          I work in 25 fps, which means that each frame represents 40ms. So I move my un-sync'd audio so that my sync point comes in the frame before where I need it - victim to the frame snapping precision, right? But then I apply a delay to the audio, set it to 100% wet signal, and then adjust it in the sub-40ms region until I'm happy with the sync. Usually, you can visually estimate using the waveform how much of 40ms you need. (Well, .001, .002 or .003 or a second, anyway.)



                          • 50. Re: How to edit video to audio with a sub-frame precision?

                            I was having the same issue and came across this video: Moving Audio Freely [Adobe Premiere Quick Tip] - YouTube


                            Basically, you can change your time scale to Show Audio Time Units which allows for much more precise movement of audio. Once they're aligned you can switch back to the standard time units.


                            Hope this helps. This was frustrating me as well!

                            • 51. Re: How to edit video to audio with a sub-frame precision?
                              matthewa75473542 Level 1

                              I've encountered these same issues repeatedly.


                              I'm working on a video where I have 4 different layers of video tracks (including lagging skype calls) over an audio bed (music/dialogue/sfx) that CANNOT be changed. It has to stay the same. Much like what you're saying for a music video where it would bad to start changing the actual audio edit by trying to cut here and there for syncing.


                              Here is one not-so-great but possible workaround I have found. Someone else may have a better one. But I haven't heard of any.


                              My video footage was 24fps in my timeline. My sequence settings were the same. 

                              I changed the sequence settings to 60fps and that lets me edit slightly more accurately. I wish I was better at explaining why this is -- other than Premiere believes there are 60 frames in the sequence, which allows me to have a few more frames to work with despite my footage being only 24fps.


                              I was able to get my audio and video syncing much closer by increasing the FPS of the sequence, even though I admit this is not ideal.

                              When I was done syncing -- I set the sequence settings back to 24fps and it didn't affect any of the syncing.


                              Someone might find a fault in my workflow and logic here, but so far that's the best workaround I've found.

                              1 person found this helpful
                              1 2 Previous Next