8 Replies Latest reply on Sep 30, 2016 10:14 AM by Neibler

    Green Screen and 3D Camera Tracker Issue

    Neibler

      Hey Guys,

       

      I'm working on this music video and the client decided to shoot some performance handheld against a green screen without informing us. The footage has proven to be a little too difficult for my current skill set. Here is a pic of the setup. I've attempted the AE 3D Camera Tracker, regular AE tracking, Mocha Planar Tracking, and even downloaded a Syntheyes demo and nothing seems to work. Either the markers are too green, or there just isn't enough detail for the trackers to pick up but the talent.

      Green.jpg

       

      The idea is to create a 3D camera so I can track in a stage set behind my actor. I've created a 3D stage in Element 3D but for the life of me I can't get good tracking data from these shots. I've also attached a sample clip in low-res to give you an idea of what I am up against.

       

      Dropbox - Green_TRACK.mov

       

      Any ideas? Any help would be greatly appreciated as we are trying to avoid re-shoots.

       

      - Nick

        • 1. Re: Green Screen and 3D Camera Tracker Issue
          Dave LaRonde Level 6

          You know what?  You might be able to use AE's plain ol', garden-variety motion tracker on this.  Have you tried it?

          • 2. Re: Green Screen and 3D Camera Tracker Issue
            Neibler Level 1

            I did try it, but unfortunately was only able to successfully track one point, which gives me y and x axis info but no rotation or anything I can use to lock him into a 3d scene...

            • 3. Re: Green Screen and 3D Camera Tracker Issue
              Dave LaRonde Level 6

              I'd delve deeper into those tracker settings if I were you.  There's an AE Motion tracker setup that'll do x-y position, rotation and scale.

               

              Twirl down those settings and look hard.

              • 4. Re: Green Screen and 3D Camera Tracker Issue
                Roei Tzoref Adobe Community Professional

                what are you settings this footage against? the question is how good is good enough? with some smoke and mirrors you can fake the floor which is the hard part. tracked in mocha in a few minutes. it could get better.

                track.gif

                • 5. Re: Green Screen and 3D Camera Tracker Issue
                  Rick Gerard Adobe Community Professional & MVP

                  This is a little 20/20 hind site for you to consider. When I do this kind of thing I always put something solid and not green like a couple of chairs. It only takes a few objects with some fixed geometry to generate a really good camera track, especially when you are doing green screen. You have to be a little more careful with the camera work and framing, but, and you can trust me on this, a few minutes of roto if you need it to clean up a key is a lot less work than hours of monkeying around with a green screen shot that's filled with trackers that serve no useful purpose for solving a camera. It's incredibly difficult for a camera tracker, any camera tracker, to get any useful information from a bunch of +'s on a flat background because there's nothing in the shot that gives you a parallax shift. Judging from the shot you provided the green screen could have only covered the center 1/3 of the screen leaving all that room on the side for a couple of folding chairs or some boxes.

                   

                  Now that you have a better idea of how to shoot a shot for a virtual set, let's talk about your problem. The best you can hope for would be to use Mocha to corner pin track the back wall using a separate spline on the same layer for each + and then corner pin track using this technique to add in your virtual set. This will give you a stationary background to work on, and then you can put the motion back in the shot later. Take it from a Syntheyes owner and user, as well as several other tracking solutions... The way this footage was shot your only hope is to stabilize the shot so that the background doesn't move, then add your virtual set and then re-introduce the motion.

                   

                  Here's the Mocha solution example using entirely different footage, but the principal is sound.

                   

                  EDIT>>>

                  Ok, I took a look at your footage, actually downloaded it and gave it a closer look. Not only is the camera moving in a spastic manner, the camera operator is zooming in and out. Without a ton of background detail you'll never even get mocha to work. There is a way to kind of save the shot. I noticed that the toe of his shoe on the right foot was basically staying in the same place on the floor. So I threw a bunch of color correction on a pre-comp to give me some detail to Motion Stabilize:

                  Screen Shot 2016-09-29 at 9.49.53 PM.png

                  Then I pre-composed and Motion tracked these points and motion stabilized the pre-comp (stabilized) using position, rotation and scale.

                  Screen Shot 2016-09-29 at 9.51.02 PM.png

                  Then I turned off the wacky levels, added pulled a key and then started experimenting with a fake 3D background eventually settling on a pair of shape layers made 3D with grid applied and positioned to kind of represent a floor. I then pre-composed the fake bg layer and added a null to the comp. I have developed a set of expressions that tie the anchor point, scale and rotation of a layer to the position, scale and rotation of the null. This set of expressions was saved as an Animation Preset because I use it all the time. You'll find it here: Dropbox - destabilize Rotation Scale.ffx

                   

                  The way you make it work is to make sure that the stabilized footage layer is named "stabilized" and apply the animation preset to a null. Then you move the CTI back to the first frame and parent all of the layers to the null. This puts the motion back in the original footage without destroying any of the keyframes and adds the motion to the other layers in the shot. So I spend about 15 min monkeying with your shot and got this, which I think looks pretty convincing

                  crazy track.gif

                  With some more careful work on the trackers and a little more work on the fake set you can probably come as close as anyone can with this shot.

                  Just for fun, her's the project file without footage. You'll have to relink your sample footage. Dropbox - crazy track.aep (note: your browser may add a .txt extension to the AEP file. Just delete it.)

                   

                  This should get you started. Next time pay a lot more attention to the camera work and put something in the set that gives you some detail to track. I would have put a bunch of cardboard boxes or chairs around the set and kept my fingers off the ZOOM lever. Don't ever zoom on a shot that you want to use as a composite unless you have very carefully planned the shot and have a lot of experience doing compositing. They hardly ever work out well.

                  1 person found this helpful
                  • 6. Re: Green Screen and 3D Camera Tracker Issue
                    Szalam Adobe Community Professional & MVP

                    Rick Gerard wrote:

                    Next time pay a lot more attention to the camera work and put something in the set that gives you some detail to track. I would have put a bunch of cardboard boxes or chairs around the set and kept my fingers off the ZOOM lever. Don't ever zoom on a shot that you want to use as a composite unless you have very carefully planned the shot and have a lot of experience doing compositing. They hardly ever work out well.

                    Remember, the OP didn't shoot this. In the initial post to this thread he said, "the client decided to shoot some performance handheld against a green screen without informing us". This is a client messup that is out of the control of our beleaguered OP.

                     

                    The good news is that it seems he came to the right place. You guys have done some great work with this awful footage!

                    • 7. Re: Green Screen and 3D Camera Tracker Issue
                      Neibler Level 1

                      This looks pretty good. Thank you for putting the time in. I have built a 3D set in Element 3D to place him in. But it looks like making the stage darker might help distract from his feet not planting firmly. Thanks for the advice!

                      • 8. Re: Green Screen and 3D Camera Tracker Issue
                        Neibler Level 1

                        Hey Rick,

                         

                        As Szalam noted, I didn't do the shoot. The client decided to go handheld before consulting me. He was instructed to shoot the chorus in multiple angles (and was even provided storyboards) and he thought this would save time

                         

                        Your tip is amazing. Great idea! I'll give it a shot. Thanks again for your time!

                         

                        - Nick