14 Replies Latest reply on Aug 14, 2016 2:52 PM by Roei Tzoref

    Hot to remove people from perspective changing clip?

    laurentc48856771 Level 1

      I am new to Premiere and After Effects (and video editing), I have looked at many tutorial on YT but i still haven't found how to remove two people on a perspective changing clip (camera raising and unveiling) shot with a dji phantom 4 (1080p 50fps) . Here is the clip: test forum - YouTube

      Can someone please explain me how to remove the two people? Thank you very much, Lorenzo

        • 1. Re: Hot to remove people from perspective changing clip?
          Roei Tzoref Adobe Community Professional

          This is advanced stuff what have we got here - all the things that could make this complicated: you got parallax shift, lens distortion, lens flare, strands of weeds over your subject and probably a few other things.  this means you would have to create several patch's that will move according to tracked data. you can try Mocha Remove module: Multi-Plane Lightsaber Remove with mocha Pro « Imagineer Systems , Tutorial: Basics of the Remove Module with mocha Pro - Part 03 - YouTube  (like I said - NOT EASY). or you could try plain old school tracking and patching with different plates: track different places and patch them up with clean substitutes and mask out and color grade to compensate for light differences (Not easy too) I tried to analyze your shot and find a clever easy solution but could not. I will mess around with it some more and maybe we have a few geniuses here, so let's wait for a while. anyways thanks for the challenge! I will give it a few tests and come back with results.

          1 person found this helpful
          • 2. Re: Hot to remove people from perspective changing clip?
            Mylenium Most Valuable Participant

            You just need to find a suitable patch of gras and shrubs and mask it out on a duplicate of the layer, move it in place and color correct it. There isn't really that much perspective involved as far as the two guys and the place where they are standing is concerned. The real challenge would be masking out the fine grass stalks in the foreground and "poking a hole" into the landscape to add your duplicate. I realyl would suggest to not get carried away and trying to be "scientific" with complicated tracking or whatever. Just use the footage you have or some photos that match the environment and patch it up. It's just going to be a lot of tedious manual work, but perfectly doable.

             

            Mylenium

            1 person found this helpful
            • 3. Re: Hot to remove people from perspective changing clip?
              laurentc48856771 Level 1

              Thanks Mylenium and Roei Tzoref for your appreciated help, however i finally fixed it in Premiere Pro duplicating the layer and masking, it might be not perfect but you can barely notice it. See at 0:41 Lexus CT200h - Top Marques - YouTube

              Thanks, Lorenzo

              1 person found this helpful
              • 4. Re: Hot to remove people from perspective changing clip?
                Roei Tzoref Adobe Community Professional

                Very Nice! and you even pulled it off in Premiere! cutting the shot 2.5 sec later really takes a lot of the work. I see mask trickery proved better than I expected. there's the grass shifting and disappearing in your shot there but it all depends how critical is your client (and how big is the screen ). never underestimate the power of mask feathering!  anyways I now had time finally for this shot and after trial an error and messing with different techniques - mask trickery proved to be the best way to go about it. I have managed to remove the people from the whole original shot and I finished this setup in 30 minutes work (but the trial and error took more ).

                 

                the setup involves just 2 duplicates of the video footage with mask in each. and only 3 keyframes on each and one with levels color correction with 2 keyframes.

                1. mask on duplicate #1 this is the background - the field (the mask is not active right now, in order to show what it's masking)

                Beginning of Timeline:

                End of Timeline:

                2. mask on duplicate #2 - the weeds. the layer is shifted to the left to cover the part that's on the people

                (the mask is not active right now, in order to show what it's masking)

                Beginning of Timeline:

                End of Timeline:

                Final Result

                ice_video_20160812-052545.gif

                closeup.gif

                 

                thanks again for the challenge. I actually learned a few thing here.

                2 people found this helpful
                • 5. Re: Hot to remove people from perspective changing clip?
                  laurentc48856771 Level 1

                  Wow thank you very much Roei Tzoref for your help and the time that you spent on it, it looks really good!! I will try to replicate it! Thanks again and since I'm a newbie, I might have some more challenges in the future!! Ciao, Lorenzo

                  • 6. Re: Hot to remove people from perspective changing clip?
                    Roei Tzoref Adobe Community Professional

                    Thank you Lorenzo! for making me think outside of the box. what you did in premiere inspired me very much and if it wasn't for you I would have tried solutions that are much more complicated.

                     

                    here is the Aep file - try to work with that. you will have to make some adjustments because I worked with a 720p version and 25frames but all the information is there

                    • 7. Re: Hot to remove people from perspective changing clip?
                      Rick Gerard Adobe Community Professional & MVP

                      I would have used Stabilize Motion to lock the two guys in the frame so I could easily work on them. This minimizes additional position, scale and mask keyframes required to do the job. Once the shot is Motion Stabilized for position, scale and rotation the layer is duplicated, pre-composed (collapse transformations turned on) then the motion is put back in the frame using this animation preset and a null. The expression connects the position of the null to the anchor point of the motion stabilized layer and also connects scale and rotation. When you parent the stabilized shot and any other elements in the scene to the null the motion will be re-introduced to the original shot and all of the elements you added to the scene will follow along with the camera move.

                       

                      It would look something like this:

                       

                      This technique will work for using the clone stamp and adding a bunch of other elements to a shot. Stabilizing for Roto work is also a useful technique because it means a lot less work for animating your masks.

                       

                      You can also use Mocha, or Mocha Pro's remove module. There are a bunch of ways to do this.

                      • 8. Re: Hot to remove people from perspective changing clip?
                        Roei Tzoref Adobe Community Professional

                        Thanks Rick. mental note to self: every time you need to make a patch on something that moves, or roto something that moves - check and see if stabilizing it first could be more efficient!

                         

                        I do agree with the idea of stabilizing first, BUT I have tried your technique and actually struggled very much to get good results. fully aware that you would never suggest something that doesn't work, I dare to suggest that I found a problem that I believe happens in your example. this is your video: Stabilizing and cloning In After Effects - YouTube  - at 04:15 - as you scrub back there is a shaky movement of the precomp layer.

                        001.gif

                         

                        it's also in my footage when I tested your method. even as you scrub through at the end of your tutorial you can see this epileptic movement happening in other segments. when I check in my attempt to follow your method, I see that moving the precomp in the current setup you suggested  creates an offset with the original track that is very noticeable. please check again and see what you make of this.

                         

                        I believe the problem is that the whole patching and masking is supposed to be in the stabilized precomp and not in the master comp. after tracking the shot in Mocha to get a more decent stabilization, I see that this is the case. this is my setup that works better, or should I say - works (where do I find the audacity???)

                         

                        Master comp

                        the Master comp contains the stabilized precomp with a null with the original motion taken from Mocha to re-introduce the motion.

                         

                        Stabilized comp

                         

                        the Stabilized comp contains the footage layer that is reverse stabilized by expression attaching to the same motion null from mocha. (Lorenzo is probably tearing his hairs by now from all of this )and on top of that a duplicate that is track matted to a solid with a mask on top of it. by moving it's position I can control both separately. and offset the layer.

                         

                        and now for the final test - here is the project file with your example and mine for comparison. I hope I haven't messed up yours, but like I suggested - I see this is happening in your video too: remove people compare.aep - Google Drive

                         

                        lastly let me say it was refreshing to see another tutorial from you and please consider to do more on workflow advice or anything. also, please forgive my feeble attempts to add humor now and then. in all seriousness, I believe there are only a handful of After Effects geniuses on the plabt, and you Sir , in my humble opinion, are one of them.

                        • 9. Re: Hot to remove people from perspective changing clip?
                          Rick Gerard Adobe Community Professional & MVP

                          I never said that this was an absolutely perfect solution. Composites seldom are. I was demonstrating very quickly an industry standard technique of working on composites and doing rotoscoping,  which is essentially what I am doing when I added the mask, that can successfully be used on shots like this one. I was not trying, nor do I ever try to do someone's work for them. I was demonstrating an efficient technique that can be very successful on certain types of shots.

                           

                          Stabilizing and then re-introducing motion has been an industry standard technique for many years.

                           

                          I spent seven minutes on this project.  Even with the highly compressed footage downloaded from YouTube footage it would be very difficult for someone who did not know exactly where to look to find the edges of the composite in a 720 P render that was played back at full resolution in real time. I showed it to my wife this morning and she was amazed when I turned off the pre-comp to show the guys standing in the field. She originally thought I was just showing her a cool drone shot. She knows what she's looking at. She's been writing and producing movies for 20 years.

                           

                          Stabilizing the footage and then reintroducing the motion does not do anything to improve or smooth out the shot. If you were to use a difference blending mode before you readjusted the position of the pre-cop you would see an exact pixel for pixel match and a black screen.  Depending upon the shot you may see a difference if you do not turn on collapse transformations. Any jitters in the original footage will be in the composite. There is nothing wrong with the way I did the masking. The quality of the patch depends entirely on the part of the footage you use for the replacement and how carefully you adjust the position.  You do have to put in a couple of keyframes when perspective and parallax changes but it's a lot easier than working without going through the stabilization process. I noticed after I had record video that I forgot to collapse transformations in the sample. That would've made my screen recording look better. I can't really tell what's going on in your example or why you are getting a poor results but I think it has something to do with position and maybe the way you tracked the footage.

                           

                          There are more accurate ways to do everything, but the time cost and the end result must be considered. At the end of my narration I mentioned Mocha. I could have also used the stabilized corner pin or the removal tool in Mocha Pro. There are a half a dozen different techniques you could use in Mocha to remove unwanted objects that you did not cover.

                           

                          If I was doing this shot on a tight budget for YouTube or standard definition broadcast I would spend a lot less time on the composite than I would if this was an IMAX production headed for theaters. The demonstrated procedure is absolutely sound for certain kinds of shots and a good and viable technique that any compositor should have in their toolbox. No one procedure is perfect for every shot.

                          • 10. Re: Hot to remove people from perspective changing clip?
                            Roei Tzoref Adobe Community Professional

                            Once again Rick you are not addressing facts. 90% of what you just wrote is Irrelevant to the issue at hand. I wonder if you even read my post. you know some say about me here that I am very analytical and well phrased - so it should not be that difficult. let me make it clear: reverse stabilization is an amazing technique and my only issue here is that using your method seems not to work - not in your video and not when I try to reproduce it. I showed you a similar method (that you actually show yourself but with corner pin) and I have sent you the project files you can see for yourself. nobody asked for perfect but it does not work the way you show it that's all I am saying.

                             

                            I am sure you know how to do this but the exact way you just showed it in your tutorial clearly show it does not stick. the layer studders. not because of the recording. it happens when I try to reproduce it. I would not bet on my reputation on it (which is clearly not as vast as yours) but this is what I am saying. if you consider yourself a forever student of this software like I am, you would get down of your pedestal and write to the point. send my regards to the mrs.

                            • 11. Re: Hot to remove people from perspective changing clip?
                              Rick Gerard Adobe Community Professional & MVP

                              Try my method again and make sure you get a good track. Apply then duplicate to stabilized layer. Before you move or mask or do anything else set the pre-comp blend mode to difference and turn on collapse transformations. You should have a completely black screen where the shots overlap. There should be a perfect match.

                               

                              Next add the null and the preset and Parent the original stabilized layer and the pre-comp to the null. The screen should stay completely black and be filled. If you are seeing some jittering then it's probably because the line between the attach point was not horizontal or vertical.

                               

                              If the blending mode keeps a perfectly black frame but you get jitters when you reposition then there are lense distortion or rolling shutter problems in the shot.

                               

                              This is a production day for me. I haven't got time download comps and footage or redo my project but I will try and doublecheck the expressions and the method to make sure there isn't something unexpected going on.

                               

                              One more little FYI. Whenever you are doing multilayer compositing the very first thing you should do is remove all lens distortion from the shots. AE has no way to use lens profiles like Lightroom does. This means you need to shoot a chart and figure out your own corrections for every lens. This will allow you to do composites right to the edge of the frame. Correcting lens distortion is very important with wide-angle lenses especially when working near the edge of the frame.

                              • 12. Re: Hot to remove people from perspective changing clip?
                                Roei Tzoref Adobe Community Professional

                                thank you Rick for you patience.

                                 

                                Try my method again and make sure you get a good track.

                                 

                                actually the tracking was rather difficult in Ae. I did select the points you selected, I assume you did not change anything in the default Tracker options. I finally did manage to get a rather decent track. rotation wasn't working well so I removed it. i think getting a perfect track is not supposed to matter much because the motion is reintroduced value to reversed value in the end so it could only mean a slight inconvenient to the whole roto process when you need maybe to add a position keyframe or adjust the mask, but not to change or studder one duplicate from another, right?

                                 

                                Apply then duplicate to stabilized layer. Before you move or mask or do anything else set the pre-comp blend mode to difference and turn on collapse transformations.You should have a completely black screen where the shots overlap. There should be a perfect match.

                                I did. and it works (BTW in your tutorial you show "classic difference"- that's really old school and as the manual states just there for compatibility with 5.0 - my first Ae I think... if that's so they really should remove that blending mode) but that's not the problem. the problem is when I slightly move the duplicate (to create the patch like you show), there is an offset and the motion isn't reintroduced properly. it happens in your video too.

                                 

                                If the blending mode keeps a perfectly black frame but you get jitters when you reposition then there are lense distortion or rolling shutter problems in the shot.

                                O.K but it's only the slightest offset in position so it's not paralax shift or anything like that. and when working in the stablized precomp like I did - it does not happen so you got that too

                                 

                                This is a production day for me. I haven't got time download comps and footage or redo my project but I will try and doublecheck the expressions and the method to make sure there isn't something unexpected going on.

                                I thank you for you time. I am about to retire too. I just want to get this right and I thank you for sharing what you know. btw this is just a project file and a very simple one. you just find the missing footage and that's it. very easy. but I understand. I will double check too

                                 

                                One more little FYI. Whenever you are doing multilayer compositing the very first thing you should do is remove all lens distortion from the shots. AE has no way to use lens profiles like Lightroom does. This means you need to shoot a chart and figure out your own corrections for every lens. This will allow you to do composites right to the edge of the frame. Correcting lens distortion is very important with wide-angle lenses especially when working near the edge of the frame.

                                thank you, I actually did this once but not with a chart but just manually with optics compensation I think...

                                • 13. Re: Hot to remove people from perspective changing clip?
                                  Rick Gerard Adobe Community Professional & MVP

                                  I found the problem. It took about 5 minutes. It was in the tracking points I selected in the original test. The workflow is valid. The high degree of compression in the source footage and the way it was tracked caused the issues.

                                   

                                  I'll redo the tutorial so that this footage can be successfully used and give a better explanation. I'll spend more time on the next one.

                                  • 14. Re: Hot to remove people from perspective changing clip?
                                    Roei Tzoref Adobe Community Professional

                                    what's that? "Roei, thank for paying attention to my tutorial and figuring out it had a problem that I repeatedly refused to acknowledge" ???

                                     

                                    GEEZ.. Rick don't make a big deal out of it, it's my pleasure don't mention it!

                                     

                                    The workflow is valid.

                                    I now believe it is, but I also believe that working the way you suggested it is more prone to problems and I will prove it to you - if you would work in a stabilized comp, and not outside with attaching the matte to the null - you would see how problematic is your track and fix it first. that's why I believe patching and roto should be done in the stabilized comp. you yourself demonstrated with that mic setup pan shot don't you remember?

                                    and also the t.v. tutorial - both with corner pin but still it's the same thing, and seeing how even you overlooked that the clone track does not stick proves my point

                                    The high degree of compression in the source footage and the way it was tracked caused the issues.

                                    I trust that is the case. anyway I tried to track it for an hour with point tracker in Ae. it is very hard to get good results. we are of course making this very difficult for ourselves because it's originally 1080p footage with 50fps and we get 720p 25fps highly compressed version. but we don't make excuses - someone smart once said: "working within the limits is the mark of a professional"

                                     

                                    I believe in this case Mocha is easily the better option for a track. and that working in the stabilized precomp is safer than working outside and attaching the matte to null. it really proves to be the safer approach and this case is an example, besides it's more comfortable to separate the movement from the stabilized (if you have many layers, you need to attach each one, render order problems could occur if you use different methods etc.)

                                     

                                    if you are still going through with this and fixing the tutorial, I would love to watch it.