13 Replies Latest reply on Jul 24, 2012 4:47 PM by Rick Gerard

    Complicated rig removal... running out of ideas...

    Lila Sundari Didi

      Hi guys,

          I am working on an aerial shot in which I need to remove some objects. I have tried many things and none of them have left the area looking real, here’s what I’ve tried:

      First, I tried to use the rotobrush tool in After Effects CS5.1 and then use an underlay of the texture that is supposed to cover it and that didn’t work due to seams in the underlay that I could not figure out how to blur together to look good, Next I tried going crazy with the clone stamp tool and attaching the clone to tracking data but the problem with that was that the source point did not attach to a tracking point (which I don’t see how you would do, so if someone could tell me, this would be ideal), after that I tried going through the footage frame by frame in photoshop with the clone stamp tool. That didn’t work because when I put them together in After Effects into a Cineon sequence, the floor looked like it was swimming, meaning that because each frame was cloned slightly differently, the clone source was again, moving and the exact pattern on the floor was moving around from frame to frame. Now, I am trying something which is a little bit more complex and may or may not work... it has been 3D tracked outside of after effects, then that tracking data was imported into after effects and attached to a null object (a sphere) and a camera was added. After that, I went to a frame where the most stuff that had to be taken out of view was in the frame and I used the clone stamp tool in Photoshop to remove the rigs that must be removed and masked out everything but the pattern of the floor. I imported that into after effects as a layer of its own and attached it to the tracking data of the 3D track, then made a mask of the layer and key framed the mask path, added a key framed curves color correction and I will add some shrubs into areas where the rigs are not able to be covered. This may not work due to the fact that there are multiple shadows on the ground area that need to stay but are currently covered by clean floor pattern but need to stay consistant with reality and how the shadows would be on the floor. Maybe I could rotobrush out the rigs that need to be removed and then place the clean floor underneath the layer which needs rig removal? What would be ideal is if I could find a way to attach a clone stamp source point to tracking data so that the source points I chose stick to the same spot as the camera moves around the set and the cloning sticks to the points where there are objects that need to be removed from the frame. As a note, this shot is a dpx sequence so it is already split up frame by frame. I am asking this question to anyone and everyone who has even the smallest idea. I have already done so much research and watched many tutorials that I thought would give me some enlightenment then didn’t because they were for shot that were far more basic than this. The camera flys around the set in a non-steady fashion and the rigs which need to be removed are sometimes behind trees and other things.

      Thanks in advance for any ideas you put forth to this project!

      Also, I am willing to work with other programs if it means I can get this done and have it look top-of-the-line.

        • 1. Re: Complicated rig removal... running out of ideas...
          Rick Gerard Adobe Community Professional & MVP

          It would really help if we saw a screenshot. There's some great tricks using Mocha. There's simple rig removal. Knowing which way to point you depends on the shot.

          • 2. Re: Complicated rig removal... running out of ideas...
            Lila Sundari Didi Level 1

            Well, I can't legally show a screen capture but I can do my best to paint the picture for you with my words. The shot is an aerial that opens looking down on some trees and we keep tracking to the left and a house with a brick deck is revealed, as that is revealed we see white deck furniture (chairs, foot rests, and a couple tables), then we can see the whole deck and all of the chairs (eight or so) but we keep on tracking to the left and the deck is slowly covered by more trees which we can see the chairs through at some points. Along with those chairs and trees, there are a number of shadows being cast left to right which either need to not be covered (shadows from trees) or be covered (shadows from furniture). Sorry if this isn't very helpful... 

            • 3. Re: Complicated rig removal... running out of ideas...
              Rick Gerard Adobe Community Professional & MVP

              Are you trying to remove the chairs? If so, with the trees it's going to be a painful experience.

              • 4. Re: Complicated rig removal... running out of ideas...
                Lila Sundari Didi Level 1

                Yes, trying to remove the chairs and trust me, it has already been a week long journey of a painful experiance but if I can get it done, then it's worth it!

                • 5. Re: Complicated rig removal... running out of ideas...
                  Mathias Möhl Level 2

                  You say it would be helpful to attach the clone stamp source point to tracking data.

                   

                  Hence, I would try to track the deck in mocha and create a stabilized precomp. Then you can paint out everything inside the precomp where it is static, i.e. no need to animate the source point.

                   

                  This tutorial gives you an idea (in particular in the end, where I remove the eyebrow of the guy. You could do this also with the clone stamp):

                  http://ae.tutsplus.com/tutorials/vfx/creating-a-tracked-head-wound/

                  • 6. Re: Complicated rig removal... running out of ideas...
                    Fuzzy Barsik Level 4

                    As far as I understand, the main issue might be in inability to track chairs through the trees, since branches and leaves shape is not so consistent and prominent for Mocha to detect on upper layer to exclude from, but quite enough to spoil the main track...

                     

                    If so, and since you don't mind working manually on every frame, I would suggest the following technique (similar to one I use sometimes for stabilising shaky timelapses): paint branches and leaves in Photoshop in a way they turned into a consistent opaque shape so as to help Mocha detect them. Then therefore create a spline around chairs for main track and a spline around those newly painted branches and leaves on upper layer to exclude from the main track.

                     

                    The above might or might not help depending on your shot and obviously whether I properly understand the issue or not...

                     

                    Since you can't place a screen shot here, you may upload sketch drawing instead so as to explain particular issues a bit better...

                    • 7. Re: Complicated rig removal... running out of ideas...
                      Rick Gerard Adobe Community Professional & MVP

                      Thanks for chiming in Mathias.

                       

                      Painting out the deck chairs is going to require careful hand work if the shot has any significant movement. Automating a clone is going to be real tough. Offsetting in time (another neat trick with Mocha made easier with Mathias' nifty AE scripts) is going to be difficult because of the foreground objects.

                       

                      This points out the great danger all VFX artists run into when we promise something before we see the shot or, worse yet, promise something without being part of the production planning. That's why the big boys hire VFX supervisors. It takes a lot of experience to know what can be done in a reasonable amount of time and on a reasonable budget.

                      • 8. Re: Complicated rig removal... running out of ideas...
                        Lila Sundari Didi Level 1

                        The tracking is all fine, I had to have it 3D tracked in a program called Boujou. Mathias, the problem isn't that it is not stabilized, just that the whole shot is a large scale rotating view of a house with a deck, I did try painting out everything where it is static, but once the camera moves, so do the chairs and therefore, we have lost covering the chairs and gone into covering random bits of the shot with nonsensical areas to cover something with. I watched your tutorial and since there is not very much overall movement, it didn't seem to help too much but I could give it a look again to see if I can try to apply something to this shot. Every frame of this shot is like the whole world was litterally picked up and shaken and swapped around, each frame is that different from the ones before and/or after.

                        As I said, the tracking is fine and going through the trees is not so much the issue at the current moment, just getting the chairs to be consistantly covered realisticly while there is nothing obstructing the view is what I am trying to focus on as of now.

                        NukeX 6.3 has a node called planar tracking, but for some reason, I can't really find anything like that in AE and I only have access to NukeX 6.2v6. I tried planar tracking in Mocha and that didn't work out very well, when the deck gets covered by the trees, the tracker gets confused and starts tracking the trees instead of the deck. I'm not quite sure why I think NukeX would be any better at it, but I sure wish I could try~

                        • 9. Re: Complicated rig removal... running out of ideas...
                          Mathias Möhl Level 2

                          If you have problems tracking it in mocha, but have already a good 3d camera track, maybe this tool also helps:

                           

                          http://aescripts.com/projection/

                           

                          It supports to create more than one "clean plate frame" in case things like shadows in the scene are changing.

                          • 10. Re: Complicated rig removal... running out of ideas...
                            Lila Sundari Didi Level 1

                            20120724_130800 copy.jpg20120724_130823 copy.jpg20120724_130833 copy.jpg    So here's some sketches to help show what the shot looks like but the last one is sort of not accurate because in the trees, there are holes where you can see some of the deck and sometimes the chairs.

                            • 11. Re: Complicated rig removal... running out of ideas...
                              Lila Sundari Didi Level 1

                              What do you mean about the "clean plate frame"?

                              • 12. Re: Complicated rig removal... running out of ideas...
                                Lila Sundari Didi Level 1

                                Well, I downloaded the script and as I was starting, it told me that the trial only works on 1080p footage, but of course just my luck would be that this is custom 2K footage (2048x1107) and I don't really want to pay for the script if it isn't going to work for this project.

                                • 13. Re: Complicated rig removal... running out of ideas...
                                  Rick Gerard Adobe Community Professional & MVP

                                  Nest your comp in a 1080P comp, use Layer>Transform>Fit Horizontally, then render. Use the render for Mocha and the script from Mathias. Now scale the comp using File>Scripts>Scale Composition. Replace the Rendered footage with the original clip and you should be good to go.