23 Replies Latest reply on Jan 19, 2010 2:21 PM by Andrew Yoole

    Weird Pulldown

    Richard G Patterson Level 1

      I have some video from a telecine transfer of 16mm film which has a weird form of pulldown where there are three split frames for every two whole frames.  In other words the four frames of film become five frames of video with the following field pattern 1ab, 1b2a, 2b3a, 3b4a, 4ab.  Is there any simple way to remove this pulldown and get 23.976 full frames?  I am thinking about using custom mattes and an offset duplicate in the timeline or writing an ad hoc plug-in, but has anyone else already solved this problem?

       

      Thanks.

      Richard Patterson

      richard@rgpost.com

        • 1. Re: Weird Pulldown
          Rick Gerard Adobe Community Professional & MVP

          Have you tried separating fields - guess...

           

          Standard film 3:2 pulldown is just that Three fields then two then three, then two. Two fields are identical, then Three fields are identical. I can't figure out your numbering system. I'm also not sure that I understand how you're counting the fields. The easiest way is to separate fields, drag your footage into the new comp icon at the bottom of the Project panel, then double the frame rate of the comp and step through it a frame at a time.

           

          The other problem with film transfers to video comes when the video is edited. You can cut anywhere in the sequence of fields so you may have one shot that is SSWWW next to a shot that is WWWSS so you end up with a sequence of fields that is SSWWWWWWSS. The only way around this is to cut up the edited video and run a separate interpretation on each shot.

           

          The last problem that shows up with film to video transfers happens when the film is transferred on a system that isn't synced (a telecine) to the video. There are a bunch of places around that just thread up the film in a projector then tape the projection adjusting the speed of the projector to eliminate flicker. If your film was not run through a telecine machine there's no automatic system to remove the jumble of frames and fields that is created by the transfer process.

           

          If you could post a section of your film, only about 60 frames, I'd be happy to take a look.

          • 2. Re: Weird Pulldown
            Richard G Patterson Level 1

            I have discovered that my problem is not what I thought it was.  The original transfer did not have 3:2 pulldown.  It does the frame rate conversion by superimposing two full frames after every four full frame transfers.  So separating fields and attempting to remove pulldown just creates interlace effects rather than curing a problem.  I gather what I need to do is let AE do the simplest possible frame rate conversion in a composition which is 23.976 fps and let the footage remain 29.97 fps and I need to make sure the first four frames in the movie are full frames so that After Effects drops every fifth frame which is a superimposition of frames 4 and 5 in the original film.

             

            It's a little academic at this point but my attempt to describe what I was seeing was based on the assumption that the options available in Interpret Footage dialog such as WSSWWW, SSWWW, etc. referred to the constitution of each frame in a five frame sequence with W for "whole" meaning that the frame consists of two fields that were in the same original frame of film and S for "split" meaning that the frame consists of a field from one frame and a field from a different adjacent frame.  In my notation I was attempting to describe the distribution of the fields in the frames as I saw them in my screwed up footage.  The number represents an original film frame and the a or b represents the first or second field for that frame.

             

            The second problem that Rick addresses is what I would call a "cadence" problem which can be caused by stopping and restarting the telecine in mid-transfer without some procedure which insures that the cadence of the pick-up matches the cadence of the transfer that was already laid down.  This can be a nightmare in removing pulldown, but it was not the source of what I was seeing.  I did notice however that Cinema Tools for FCP claims to be able to detect cadence problems and adjust for them when it does what it calls "reverse telecine."

            • 3. Re: Weird Pulldown
              Rick Gerard Adobe Community Professional & MVP

              Cinema tools does a fair job but it too gets confused sometimes. That's why, for years, when I shot commercials on 35mm for television I always shot at 29.97 fps and transferred without pulldown. I always disliked the look of 3:2 and it can be a real mess when editing or compositing.

              • 4. Re: Weird Pulldown
                Richard G Patterson Level 1

                Actually I am still having problems and I am confused by what After Effects is doing to the footage.  If I set interpret footage to 30 fps and set separate fields to Off and put the footage into a 24 fps comp, I get split field interlace effects in the comp.  I don't understand why it is spliting the fields apart at all rather than just dealing with the full frames.  Is there a setting that determines how the composition attempts to adjust the frame rate of footage?  How can I make it just drop every fifth frame?

                • 5. Re: Weird Pulldown
                  Rick Gerard Adobe Community Professional & MVP

                  That's what I would expect to happen. If you don't separate fields and you've an uneven number of fields then some of the frames will have mis matched fields. Forcing the footage to a different frame rate only effects the playback rate, it doesn't do anything with the fields. If you put footage of one frame rate in a comp of another frame rate then you'll end up with some weird blending of frames in the comp. This has to happen because you're dividing up a fixed number of frames into a different number of frames.

                   

                  If your footage has fouled up or mixed frames when you don't separate fields then dropping it in a comp with a different frame rate isn't going to fix the problem. It's just going to make it worse.

                   

                  It would really help to see a few seconds of the original footage. Maybe then I could come up with an idea for a solution.

                   

                  The way to remove every fifth frame is to posterize time in a comp with a frame rate that matches your video (29.97). I'd have to think about the settings for a while and probably AE open to give you the settings.

                  • 6. Re: Weird Pulldown
                    Richard G Patterson Level 1

                    Here's a 40 frame piece that I believe shows what I am dealing with.  It has four full film frames followed by a fifth which is a blend of two full film frames. There are no interlace artifacts visible in the blended frame.  It seems to be just a 50:50 blend of the previous and subsequent film frame.

                    • 7. Re: Weird Pulldown
                      Andrew Yoole MVP & Adobe Community Professional

                      You're correct that the clip runs as four frames-clean, with each fifth frame a mixed frame.  It's not pulldown and there is no interlacing.

                       

                      Rather than a true telecine, the footage appears to be a straight capture of projected film off a screen or wall - the sound implies that too.  If that's the case, the issue may "drift" over time, making it difficult to remove with any automated process.

                       

                      I stepped through your extract file in QT Pro and manually cut each 5th (mixed frame) out.  The resulting clip seems fluid - the motion doesn't appear to be affected by the missing frames - but of course it runs 20 percent faster.

                      • 8. Re: Weird Pulldown
                        Richard G Patterson Level 1

                        If it does not drift, surely there is a way to make AE drop every fifth frame.  What is the best way to do this?

                         

                        If if does drift, I'll chop it up into short clips to process separately.  I don't really want to use QT Pro to manually cut every fifth frame from an hour and a half of footage.

                         

                        One other thing that is puzzling me is that QT Pro seems to split fields when I open this footage in it.  I see interlace artifacts when I open the DV footage in QT Pro and step through it even though I am trying not to do anything other than just view it as it is.  I have high quality selected for the video in QT.  Can  you think of what may be causing this.  I'm afraid to edit it and save it out of QT because of this.

                        • 9. Re: Weird Pulldown
                          Rick Gerard Adobe Community Professional & MVP

                          I took a close look at your footage. I followed my own advice for analyzing problem footage. I let AE do the interpreting, drug the footage into the new comp icon and then doubled the frame rate of the comp so I could look at each field and check the motion.

                           

                          AE thinks that the footage is NTSC DV, separating lower first (the default guess for this frame size) with 29.97 FPS for the frame rate. That's a good guess because of the frame size (720 pixels by 480 pixels) and the codec, but it turns out that it's not the correct interpretation.

                           

                          Your footage is comprised of pairs of identical fields. In other words, it's progressive, not interlaced. If it were interlaced and had pull down you would see some matched pairs but then you'd see one pair that is not matched. IOW there would be two fields for one frame then one field from a frame followed by another field from the next frame, then two fields from the same frame. What you have is identical pairs of fields.

                           

                          You can reverse the field order with no change in the footage. If it were interlaced each field would be different. Reversing the field order has no effect on the motion and never reveals an odd pair of fields. If the footage were interlaced the action would be back and forth with reversed field order. I hope you followed that. It's kind of confusing to talk about but it's easy to see when you step through the footage.

                           

                          This means that the mushed together frames that you're seeing are not the result of a pulldown problem. There cannot be pulldown if there are no fields. The mushed together frames are just that, a blend of one frame of film with another frame. Pull them out and there will be a hole in time... IOW, the action won't be smooth. This tells me that the footage was not transferred by a telecine, but by a filming a projector. I expected as much because the footage looks like somebody filmed a screen with a camera. The overlapping (mushed together) frames will probably drift over time because the projector motor would not be exactly in sync with the video camera. This footage may have been interlaced at one time, but it is now definitely not interlaced.

                           

                          You could remove every mushed frame but you would not end up with smooth action. It probably wouldn't be noticeable at full playback speed, but there would be holes in time or places where the movement from one frame to another would not be linear or smooth or fluid. I don't think you'd gain anything. The mushed together frames are not noticeable at playback now.

                           

                          I'm not sure what you're trying to accomplish with this footage. The quality is very poor, the exposure and gamma need some work, and you may be able to remove some of the noise or sharpen up the image a bit. That's about it. I don't see any advantage in removing the blended frames.

                           

                          If you have some other purpose in mind please let me know and I'll try and help.

                          • 10. Re: Weird Pulldown
                            Andrew Yoole MVP & Adobe Community Professional

                            Richard G Patterson wrote:

                             

                            If it does not drift, surely there is a way to make AE drop every fifth frame.  What is the best way to do this?

                             

                            If if does drift, I'll chop it up into short clips to process separately.  I don't really want to use QT Pro to manually cut every fifth frame from an hour and a half of footage.

                            You can extract every fifth frame fairly easily.  Make sure your footage is interpreted as progressive/non-interlaced.  Drag the footage into a new comp, and trim the layer so that the first mushed frame is frame 5 (frames 1-4 are clean frames).  Apply a time stretch of 80%, and then make sure all frame blending options for the layer are switched off.   AE simply extracts every fifth frame in this process.  As Rick and I have mentioned, if this stuff is just shot with a conventional projector, the mushed-frame will almost certainly drift.

                             

                            The issue you are now presented with is that the footage is now running 20% faster than it should.  If it's your goal to do some frame-by-frame masking, colouring, rotoscoping or something, perhaps this is all worth the trouble, and you can re-adjust the speed after your work is applied.  But as Rick said, the footage plays fine as-is, so if you don't have any frame-accurate compositing work to do, I'd leave it alone.

                             

                            Richard G Patterson wrote:

                             

                            One other thing that is puzzling me is that QT Pro seems to split fields when I open this footage in it.  I see interlace artifacts when I open the DV footage in QT Pro and step through it even though I am trying not to do anything other than just view it as it is.  I have high quality selected for the video in QT.  Can  you think of what may be causing this.  I'm afraid to edit it and save it out of QT because of this.

                            This is of concern.  QT Pro cannot add interlacing to footage.  If you are seeing true interlace artifacts in the original material, then you've somehow deinterlaced the segment that you've posted here, rendering our advice useless.  Are you able to post a screen grab or single frame of the interlace artifacts for us to see?

                            • 11. Re: Weird Pulldown
                              Richard G Patterson Level 1

                              Andrew's analysis is the same as mine and I believe is correct.  The video is actually a transfer from a VHS cassette to DV so that I could import it with a Firewire connection to the DV deck, and it is clear to me that the original film to video transfer was done with some kind of setup which converted from 24 to 30 fps by adding a frame after every fourth film frame.  The new frame is simply a blend of the two adjacent film frames.

                               

                              The reason I am trying to remove the blended frames is so that I can clean it up a bit as 24 fps and then make a 24p DVD so that when the viewer steps through it one frame at a time he never sees a frame created from fields of adjacent frames with interlace artifacts.  This is the right way to make a DVD from a film original.  Even the video playback will be better if the 3:2 pulldown is added during the playback of the 23.976 fps video. 

                               

                              There is obviously a limit to how much restoration can be done with the footage.  It was originally 16mm black and white reversal footage of a high school football game shot in 1959.  I am preparing the DVD for a 50th class reunion.

                               

                              So far as the peculiar display of the movie in QT Player is concerned I am attaching two screen grabs.  The first shows a frame of QT Player playing the original DV capture of the transfer.  Interlace artifacts are clearly visible.  The second is an animation (lossless grayscale) QT movie made by running the DV footage through AE as 29.97 footage in a 29.97 comp and outputting the QT with Animation compression.  You can see the blended frame, but without the interlace artifact.

                               

                              I think Andrew has provided the solution to my dilemma.  I just need to start on a blended frame instead of a clean frame which I had assumed.

                              • 12. Re: Weird Pulldown
                                Rick Gerard Adobe Community Professional & MVP

                                The system used to transfer the film to tape didn't add a frame or anything fancy. All that happened was that a camera was pointed at a wall, the projector was started then the speed control on the projector was turned until the film stopped flickering in the viewfinder and they pressed record. The blended frames you see are just two frames of film that were captured by one frame of video. Somewhere in the process of capturing the VHS to DV you lost the fields.

                                 

                                Also, I don't think you completely understand the workings of a 24P dvd... Unless you have a digital connection to a digital monitor through a DVD player that will truly output 24p video to a monitor that will truly play 24P video you'll get 3:2 pull down introduced by the DVD player through the component or composite video connection to the monitor. Stepping through the DVD a frame at a time won't give you clean frames on every frame unless all the ducks line up perfectly. Even if you remove the blended frames, you'll end up with some missing moments in time where the action jumps. If the DVD/Monitor connection is through composite video then you'll end up with a bunch of mushed together frames because the DVD player must introduce a pulldown scheme to deliver composite or component video because composite or component video always runs at 29.97 fps with fields. There's no other way to get the signal to work.

                                 

                                The best thing you could do to this video would be to render it out as 30P (don't introduce fields) and leave it alone. This will give more original frames, keep the speed close to the original projected speed, and any DVD player / monitor combination will give you fill frames.

                                 

                                There's also a flaw in the time shift solution. When you play at 80% that means that each frame lasts 80% of the normal time. It's only a matter of time before you get a blended frame. With frame blending turned off you'll get one or the other rather than a blended one, but the odds are against you for an easy solution. All you're doing is cramming more frames into less time, you're not mathematically skipping every 5th frame. The math isn't completely accurate either. The speed would be 5/6 or 83.33333%.

                                 

                                If it were my project, and I'm a real perfectionist, I'd make the picture look as good as I could, then render out a lossless (Animation Codec) or nearly lossless (PJPEG) compressed movie, drop it in my DVD authoring program, build a nice menu, and render out for 30P DVD. Your class mates will love you for it and be amazed. All the fussing trying to get the mushed frames out of the show won't buy you a thing.

                                • 13. Re: Weird Pulldown
                                  Richard G Patterson Level 1

                                  Now as I step through it I can see the drift and I believe I understand the source of my problem.  It is a film to video transfer setup which involves a 16mm projector, a small rear projection screen and a video camera.  I can remember when these were often used for transferring home movies.  I'm a little surprised by how much drift there is, but obviously such a set up did not use a sophisticated projector, so who knows how fast it was really running. There was never any sound track to use in judging the quality of the transfer. 

                                   

                                  My only option, I fear, is to live with the problem or step through the movie in QT Pro and cut out all the offending frames.  Then I just need to coerce some application into running it at the right speed.  I would suspect AE can do this, but I guess I should check it on a sample before I go through the whole thing and laboriously cut all the offending frames.

                                   

                                  Thanks for  your help.

                                  • 14. Re: Weird Pulldown
                                    Andrew Yoole MVP & Adobe Community Professional

                                    Based on the new images you've posted, most of my and Rick's previous advice doesn't necessarily apply.  The DV footage is definitely interlaced, but all the samples you've provided otherwise are not.  Which means, somewhere along the line you've deinterlaced the example footage.

                                     

                                    If you have QT Pro, just cut and paste 50 frames into a new Quicktime file, save as standalone file and post that here.  To export the sample footage from AE (if you don't have Quicktime Pro) make sure the footage is interpreted as progressive/non-interlaced, then render the segment progressive/non-interlaced.  While this is not the correct procedure for your end product, it will leave the fields unmangled for us to assess.

                                     

                                    Richard G Patterson wrote:

                                     

                                    The reason I am trying to remove the blended frames is so that I can clean it up a bit as 24 fps and then make a 24p DVD so that when the viewer steps through it one frame at a time he never sees a frame created from fields of adjacent frames with interlace artifacts.  This is the right way to make a DVD from a film original.  Even the video playback will be better if the 3:2 pulldown is added during the playback of the 23.976 fps video. 

                                     

                                    To be honest, I don't agree with your plan, for a number of reasons.  First and foremost, without an accurate, calibrated telecine transfer of the film footage, it's very unlikely you'll end up with an accurate 24fps master.  You comment "this is the right way to make a DVD from a film original", but you don't have a film original.  You have a DV dub of a VHS video of a film projector on a screen - it's very unlikely that 24p can be extracted from your source.

                                     

                                    The solution I posted earlier, using Time Stretch, won't provide you with a 24p master anyway.  Once you remove every 5th frame, you need to re-time the footage or it will run 20% too fast.  That means re-stretching it, which will either add more mixed frames, require pixel interpolation, or add duplicate frames or pulldown.

                                     

                                    Secondly, how many people do you know that use 24p DVD players?  And of those, how many have TVs that support the standard, and have them set up correctly?  And how many of your viewers will know or recognize the distinction between 24p and standard NTSC?  It seems that you are going to an inordinate amount of time and trouble to produce content that I doubt your audience will appreciate, unless your class reunion is actually the Cinematography class of '59. 

                                     

                                    Personally, I'd accept the footage for what it is, grade it as well as possible and make a standard NTSC DVD that all users will be comfortable viewing.   Just my 2 cents.

                                     

                                    EDIT:  Whoops, Rick beat me to the punch again!  Seems we're all thinking down the same line anyways.

                                    • 15. Re: Weird Pulldown
                                      Richard G Patterson Level 1

                                      I think I do understand something about a DVD made with 24 fps film based on the experience we had making DVD demo reels of film shots and based on a DVD I made to restore a film documentary that was 25 years old.  In both these cases it was my experience that using 23.976 video as the source material for the DVD could not only save data space on the DVD but also permit the user to get clean frames when he stepped through a shot frame by frame on a DVD player that was equipped to do this.  Yes, when it is playing a full speed, the DVD player adds 3:2 pulldown in order to output 29.97 fps video to the monitor, but if you pause it you never see interlace or split field artifacts.  Normally viewers don't care about clean frames when they step through a DVD, but visual effects supervisors certainly did and I suspect someone trying to see what he did during a touchdown play 50 years ago (like me) he will be happier if every frame is clean.  Part of my plan in making the DVD is to add occasional "instant replay slow motion" of important plays.

                                       

                                      Step through a sequence in any commercial DVD of a feature film and you will never see any split field frames.

                                       

                                      I am also still not convinced that removing the blended frames will cause any of the action to be lost.  So far as I can tell every time there is a blended frame, the clean frames exist before and after the blended frame.  Thinking about the nature of the transfer set up finally makes it clear to me why some blends seem more noticeable than others and why they "drift" as the movie plays.  But the frame rate difference between the video and the film would never mean that you got a blended frame without getting the clean frames on each side of it.  At least I don't believe it would.  Perhaps I will discover I am mistaken on this.  I'd do the math, but you'd have to know the actual running speed of the projector to predict whether you would ever lose an entire frame.  I can see how you might get some flicker, but I find flicker less objectionable than recurring blurriness.

                                       


                                      • 16. Re: Weird Pulldown
                                        Rick Gerard Adobe Community Professional & MVP

                                        So I spent some more time with your footage. It looks like if you start on the first good frame of the clip you posted and set the speed to 80 with frame blending off you'll get clean frames for all of that clip. I don't know how long that would last. If you want to bring the speed back here's what you do. Set up the comp at 29.97, fix the frames, render the footage without fields. It looks like my math was wrong, but then I've been working for more than 20 hours straight without sleep. I must not be thinking clearly. (only one more render and I'm off to bed)

                                         

                                        Each clip with have to be checked, but it's going to be faster than deleting the frames by hand. Just remember to re-interpret the footage and turn field separation off.

                                         

                                        When you create your DVD's interpret the footage as 24P and you should be fine and the playback speed will be closer to the original projector speed. You just can't do it by dropping the footage in a 24 fps comp. FPS doesn't matter anyway as long as you get a string of good frames.

                                         

                                        BTW, My DVD player (not high end but 24P capable according to the sticker) does show funky frames when you play slow mo 24p footage through the composite cable. It always freezes on a clean frame though. I think your mileage may vary here because there are differences in the hardware.

                                        • 17. Re: Weird Pulldown
                                          Andrew Yoole MVP & Adobe Community Professional

                                          Regardless of all discussed, based on the still you posted I still believe your DV material is interlaced, making much of this discussion potentially moot.  If you can post an example of a segment like before with the interlacing intact it will be much easier to assess your options.

                                          • 18. Re: Weird Pulldown
                                            Richard G Patterson Level 1

                                            Here's an extract from the original firewire capture of the DV transfer where you can see the WSSSW interlace pattern.

                                             

                                            One of the things that has caused some of my confusion is the fact that if I open the original DV movie from the capture in QT and step through it I can see the interlace as in this sample, but if I copy a few frames out of it and paste them into a new QT movie, the interlace disappears and I have the blended frames.  The only way I was able to get this extract was to cut off the before and after sections in the original movie using QT and then save the movie as a separate free standing movie.  Somehow in the copy and paste operation QT changes its mind about the make up of the frames.

                                             

                                            I appreciate all your efforts to help me with what you all agree is a problem not worth solving. 

                                            • 19. Re: Weird Pulldown
                                              Richard G Patterson Level 1

                                              Trying again to post the file.

                                              • 20. Re: Weird Pulldown
                                                Andrew Yoole MVP & Adobe Community Professional

                                                Okay, thanks for the upload.  Now we know the footage is interlaced it changes things a bit.

                                                 

                                                I managed to convert your footage to a clean 24p version, but I'll re-stress that, because of the drifting projector speed, it's very likely this will be an almost impossible task.  It's nice and easy with 50 frames of footage, but depending on the amount of projector drift and its consistency, it could take you weeks to recalibrate the whole segment, if it's possible at all.

                                                 

                                                Here's the process I used:

                                                 

                                                Import original DV footage and interpret as 30fps, lower field first.

                                                Place into 60fps comp and trim layer-start so that the mushed frame is always the fifth frame in the sequence.

                                                Apply 80% time stretch to layer.

                                                Render a new progressive/non-interlaced Quicktime file (preferably lossless codec) ensuring there is no frame blending active.

                                                Import rendered file to AE and interpret as progressive/non-interlaced 60fps.

                                                Place into 24fps comp and apply 125% time stretch.

                                                Render final output as 24fps progressive/non-interlaced, ensuring there is no frame blending active.

                                                 

                                                 

                                                The final result is a 24p file that has every fifth (mushed) field of the original material removed.  While I have strong doubts that the process will work reliably over an extended piece of footage, I guess it can't hurt to try.

                                                • 21. Re: Weird Pulldown
                                                  Richard G Patterson Level 1

                                                  Thanks I may try this, but it seems to me the net result is going to be frames which are made of single fields interpolated to produce full frames.

                                                   

                                                  I am afraid the drift is sufficient that this will only work for a few seconds at a time.  I had thought I might chop the whole football game up into individual plays and fix each one separately, but I fear the sequence of bad frames changes even more often than that.

                                                   

                                                  I think you can all declare victory and quit the field on this one.  If I do go ahead with my attempts to clean it up, I'll let you know if I am successful.

                                                   

                                                  Thanks again for all your time and effort.  It is very encouraging to know there are still communities of people who support each other in this way.

                                                  • 22. Re: Weird Pulldown
                                                    Richard G Patterson Level 1

                                                    Here's a final irony.  After viewing it for a while I think the footage I am working with was shot at 18 fps instead of 24 in order to preserve film.  There was never any sound and the only purpose of shooting the film was for coaching review.  It's hard for me to tell, but the speed of everything seems a bit more natural if I interpret it as 18 fps (after the blended frames have been removed).

                                                    • 23. Re: Weird Pulldown
                                                      Andrew Yoole MVP & Adobe Community Professional

                                                      Very possible.  18fps was, of course, the standard for home film cameras when shooting without sound, so it's highly likely.