4 Replies Latest reply on Feb 15, 2016 10:52 AM by Paul Michelle

    How to get AE to remember where 3D assets are placed for each shot composition?


      I will try to explain this in simple terms as I am a newbie..


      Let us say we have live action camera footage outdoors in an open space of empty land. Two people are filmed talking to each other standing in an interview side by side, sort of in a V-angle. The camera is filming whoever is the current active speaker and moves from one speaker to the other speaker. The empty field in the background can be seen. What if one wanted to import a 3D model of a tree and a 3D model of a  park bench and composite them in the background behind the first person and import a 3D model of a garbage bin and a 3D model of a street sign and composite those behind the second person.


      Question1: The question here is, what is the industry standard workflow? Does the compositor need to follow the match/track/import model process for every shot in the scene in order to add the relevant specific 3d assets EVERY TIME? Or is there a way to ask After Effects' memory/brain to remember and 'stick' the assets with 'tape' or 'glue' to the field's ground surface somehow, so they can simply be imported to the scene when the relevant shot with the relevant person is shown? i.e. Person1 has a background of a tree and a park bench and Person2 has a garbage bin and a street sign. What is the industry standard common workflow for After Effects to 'remember' where to place the 3d assets? Is there a name for the 'remember where my 3D assets are placed' process I am talking about here in Compositing language/lingo?


      Question 2: What if the two people walked 10 feet away from that area and then turned around and walked back to the original area where they were standing, but instead of stopping there, they continued walking - i.e. a continuous walk. How can we setup After Effects to remember and show the four 3D assets/models in the relevant places as the two people walked passed them where they were originally placed?

        • 1. Re: How to get AE to remember where 3D assets are placed for each shot composition?
          Mylenium Most Valuable Participant

          This is no different than your other question - I'm afraid you have a completely wrong understanding how this stuff works. Nobody would bother to re-create a full 3D-environment when it's not necessary. Static items can easily be placed in equally static shots by eyeball on a shot-by-shot basis. Everything else requires some sort of 3D tracking, but that is entirely dependent on the shot having been planned and set up correctly with markers, on-set measurements, additional reference photos and so on, but would most of the time be tackled per shot all the same. Likewise, the scene reconstruction you alluded to would require proper planning, greenscreen work, additional set and environment photos, possibly lots of complimentary 3D work. And as I said in the otehr thread, too - nobody bothers to attempt such complex stuff in AE. Sure you can place a bench and a tree in the background and make it look reasonably real, regardless of they are cutout photos or 3D models imported via plug-ins, but detailed, moving shots with secondary object motion and VFX? Forget it! That's where AE is useless as an euter on an ox. Such stuff is and will be for a while the domain of Nuke and all those tools around it, including dedicated 3D tracking programs, photogrammetry tools, 3D programs and even specialized tools like Mari for texture creation.



          • 2. Re: How to get AE to remember where 3D assets are placed for each shot composition?
            Rick Gerard Adobe Community Professional & MVP

            If you want to add elements to a scene and make them respond to the camera you only have two options. The first is to run a Camera Track on the scene, establish a ground plane and origin and then insert your elements. The second option is to use motion control equipment on your camera and then use that motion control data to control the position of a 3D camera in a 3D app. There is no direct way of importing tracking data from a motion control rig directly into After Effects. I don't know anything about your production but I doubt that you are spending many thousands of dollars a day for a motion control rig to shoot your shots. The only other option is to Camera Track your scene. Since no camera tracking data is included in any video file you must perform the task of Camera Tracking, establishing a ground plane and inserting reference layers for each and every shot.


            AE's camera has no lens distortion so if you are shooting with a camera that has a lot of lens distortion it's a good idea to try and remove as much of that distortion as you can before Camera Tracking. This goes for Motion Control machinery also. If the lens is pretty much a normal lens with minimal distortion you can get away with just tracking the shot.


            AE's Camera Tracker does a very good job tracking shots that have a fair amount of fixed geometry in the shot and that geometry has a good deal of detail. You cannot Camera Track something like a telephoto shot of a huge crowd of people walking down a sidewalk because everything in the shot is moving. There is no fixed geometry. You cannot walk along a beach and shoot a shot of the waves coming in and get a successful Camera Track unless you include enough of the shoreline or other things that are not moving to give the tracker some fixed geometry to track. You cannot track a shot of a wheat field on a windy day with a blue sky and a combine reaping wheat because the wheat is all blowing in the wind, the combine is moving and there is no detail in the sky.


            Now that we've got that out of the way, with the right kind of shot AE's camera tracker is very good at generating an accurate camera that mimics the movement of the camera used to shoot the original video. With a good track you can insert 3D layers that are attacked to any plane (surface) of the geometry in the shot that you can use to generate a good target. The reference objects will stick right where they should stick. If you want to some other reference plane object somewhere between any of the surfaces you have identified in the shot then you can easily add another 3D layer, hold down the Shift key and temporarily parent the layer to one of the reference planes. This will ship the new 3D object to the same position as the reference layer. You then use AE's other views to position the layer between the reference layers. IF YOU want to use the Same Shot for multiple projects then you can duplicate the comp in the Project Panel and change the duplicate comp. Once you have added the camera and some reference layers you can delete the Camera Tracker effect. After the cameras and reference layers are added there is no further use for the effect in your project.


            If you want to add 3D elements to the scene when you have your camera and reference layers setup properly you can export the project to C4D and this will generate a 3D project with AE's camera, lights and reference planes in the right place in C4D. You can then use the reference planes to position your 3D object to the scene.


            Back in AE if you have obstructions in the scene, like your actors walking between the 3D object or the layers you have added then you use a copy of the footage layer or a solid and you rotoscope the layer to hide the parts of the 3D object or layer you need to fix so the illusion is preserved.


            If you need to generate a 3D model of the objects in the Scene you can do with with 3RD party software like Syntheses. These 3D models can the be added to your AE comp by first bringing the Syntheses info into a 3D app. In most cases, with the shots you are describing, simple camera tracking will do the job.


            Here's two short videos demonstrating Camera Tracking and masking and then Camera tracking to C4D Lite and adding and animating a 3D object. They are very short and not detailed but you'll get the idea.


            Sometimes there is another option and that is to do the whole thing in 2D using Mocha or Stabilize Motion in AE, but that's another discussion. The right technique depends entirely on the shot you are using. All of the techniques require all of the work of tracking and placing reference layers for each shot you want to track.

            • 3. Re: How to get AE to remember where 3D assets are placed for each shot composition?
              trey38647 Level 1

              Thank you gentlemen; I always appreciate your replies and for bearing with my lack of knowledge in this area as a newbie learning.


              Mr Gerard, I appreciate your quite detailed reply. Congrats on your awards too! Maybe one day I will step up to the ranks of you experienced folks after learning and gaining experience myself!

              • 4. Re: How to get AE to remember where 3D assets are placed for each shot composition?
                Paul Michelle Adobe Community Professional

                One other thing I think you are misunderstanding In your description of the two people talking and the camera switching back and forth:  The camera is not switching back and forth..it is two long shots of each person, the camera never leaves them, the illusion of the camera cutting back and forth is created in editing.  So in that scenario there would be no need for AE to remember any position each time the camera cuts back and forth.  You would do any inserting and tracking for the entire length of the shot and then cut it up in editing.

                1 person found this helpful