3 Replies Latest reply on Jan 21, 2012 10:38 AM by Rick Gerard

    Mapping motion from video to final image

    LeRig

      Intro

      I'm not sure what kind of software I would use for this but I figured after effects would be the best to do most of it - Hopefully it can!

       

      I have posted this thread over on the photoshop forums but decided even if I could get it to work it might not even be that good.

      http://forums.adobe.com/thread/951032

      Basically I intended to import video frames into photoshop and use the smart object - stack mode - range

      heres an description of what this is

      http://help.adobe.com/en_US/Photoshop/11.0/WSA1E7602F-1D34-4ecf-B2FE-2BB344D6937Ca.html

       

      But even if I could do that in photoshop, I doubt the results would be useful as the range stack mode doesnt seem to show purely movement, it also shows colours and brightnesses so for example in the thread shown above I have uploaded an example I've made from the first 500 frames of the footage, there are 2 people walking with white shopping bags and these show really bright in the final image. - I might as well do a long exposure photo as this would show the same thing! - but they dont purely show movement.

       

      What I want to achieve

      I want to finally end up with a 3D model (in 3ds max, rhino, maya or sketchup etc...) which physically represents peoples movement through an area on the highstreet. - This is generated architecture for my project at University.

       

      The Steps involved (modify accordingly if you believe these can be improved)

      1 - Record a decent length video in 1080p from a camera mounted on a tripod and from a view as central and high up as possible which can be corrected as much as possible to be in plan view.

       

      2 - Process the video in after effects to track the movement of the pedestrians walking through the area.

      something that it could do is; a) detect distinctive movement (pedestrians walking), b) process that movement with a rule such as; [If motion is detected] [record the areas of motion throughout the video footage] [render those motion paths to an image] [save the image as a file]

       

      3 - In rhino or 3DS max I could (maybe?) then open the image which after effects has recorder and (with help!) write a script that could convert each pixels greyscale value into a z axis (x and y are the location of the pixel in the image) and use these 3 axis to make a point in space.

       

      4 - I can then make all these points into a surface and smooth it out.

       

      I'm hoping to end up with a model like this; (sketched section representation)

      (I'll upload this soon)

       

      What do people think about the process I'm suggesting, is it possible and can it be improved?

      Really appreciate the help!

       

      p.s. I have no clue how to use after effects or rhino 3d, I have basic knowledge of 3DS max and I'm very good in sketchup and photoshop.

      Luke.

        • 1. Re: Mapping motion from video to final image
          Dave LaRonde Level 6

          This sounds like you need motion capture capabilities rather than video.  Think "Lord of the Rings" and Gollum.  Or to bring it to the present day, "The Adventures of Tin Tin".  Both used motion capture to generate the actions of 3D models.

          • 2. Re: Mapping motion from video to final image
            Mylenium Most Valuable Participant

            Nah, forget it. What you describe is hardcore 3D tracking and that of the analytic kind requiring multiple cameras. There is no way whatsoever to reconstruct such complex data from just one camera mounted randomly somewhere. In that case you could just as well draw some random paths freehand and it would be just as correct/ incorrect - whichever you prefer to a call it...

             

            Mylenium

            • 3. Re: Mapping motion from video to final image
              Rick Gerard Adobe Community Professional & MVP

              If you have a locked off camera looking at a street scene and you want to track the movement of people on the ground you'll only have luck if the street scene isn't very crowded. You could try After Effects Point tracker to track individuals one at a time, or Mocha, to track people one at a time. The other option would be to use Camera Tracker from the Foundry.

               

              All of these will just give you the ability to tie a layer's position to movement. Then you'd have to work up an expression based on velocity to control opacity to automatically detect when the layer was moving. Once that was done, the layer's movement could be turned into a path which was applied to another layer. The path could be stroked with a color and the in and out points of the stroke tied to the movement of the layer with another expression.

               

              If I new better what you were trying to achieve or you had an example maybe I could give you better ideas. The only thing I can say for shure is that you'll need to do a bunch of tracking, a bunch of tweaking to those tracks and a bunch of experimentation with expressions to get what I think you want. If what you want is accurate data based on movement then Mylenium's right, a single camera and AE isn't the right tool. You'll need to do some research on motion capture.