8 Replies Latest reply on Feb 27, 2012 2:12 PM by ZentaiKitty

    Problems tracking face in patterned zentai suit




      I'm using AE CS5 and Mocha 2.2. I am trying to track face images to various clips of a person doing things in a black and white patterned zentai suit. For small movements, this seems to work fine. However, for movements like the head dropping to the chest or the person dancing around in circles, the tracker in both AE and Mocha seems to frequently lose track of the facial area of the zentai'ed person. It looks like maybe some movements, like when the patterned chin moves towards the patterned chest, for example, cause the tracker to lose the ability to differentiate between the two or...? Does anyone have any experience with this type of issue, any tips/tricks to ease the tracking pain?



      Thanks much for any and all assists,



        • 1. Re: Problems tracking face in patterned zentai suit
          Rick Gerard Adobe Community Professional & MVP

          Boy you picked a difficult subject to track. The patterns are so similar that it's going to take a bunch of manual adjustment to get Mocha to follow the face and bring in perspective. The pattern is just to repetitive to allow automatic tracking. You can adjust the track in mocha fairly easily, it's just going to take some time.

          • 2. Re: Problems tracking face in patterned zentai suit
            ZentaiKitty Level 1

            Thanks for the quick reply, Rick.


            Would maybe filming with something like a piece of bright red tape where the nose, eyes and mouth (these will sometimes be 1 graphic, sometimes mix n match separates) should track to make the Mocha or AE tracking sans tons of manual adjustments more workable?




            • 3. Re: Problems tracking face in patterned zentai suit
              Mylenium Most Valuable Participant

              The biggest problem is the repetitions of the pattern which will throw off pretty much any tracker - they all try to find a unique structure in the image and latch on to that. For the AE tracker adding some tape markers might work to a degree, for mocha not so much since it operates on detecting areas and edges, not isolated spots. For that case you would really need to have the face as a differently colored area. Still, even then, as soon as the head goes down you'd obscure things again, requiring manual adjustments. There's simply not going to be a perfect solution for this...



              • 4. Re: Problems tracking face in patterned zentai suit
                ZentaiKitty Level 1

                Thanks, Mylenium


                Shoot. I kinda figured I'd probably be heading towards a lot of manual adjustments, but had fingers crossed for a "magic tip." ; - ) I'll get some footage with tracking tape and see how well it works in AE, but prepare to extend needed edit time or see about a change of suit.




                • 5. Re: Problems tracking face in patterned zentai suit
                  Rick Gerard Adobe Community Professional & MVP

                  The magic tip is pre-production planning. I recently fixed up a project that took 2 days of hand work to correct a production problem caused by shooting first and asking questions later. A few weeks later we shot another scene, but this time, the shot was planned with post in mind. Took 2 hours to do basically the same thing.

                  • 6. Re: Problems tracking face in patterned zentai suit
                    ZentaiKitty Level 1

                    Well, luckily we haven't done much yet besides a few basic poses, like the one above. So, I think we're still safely enough in the discovery stage for making changes to the bits involved in the scenes. Would you have any advice for getting this particular suit to work better? (It is preferred that it not be changed out, and I'm also not sure if another zentai, even a solid one, would not simply cause the same problem, unless it had some sort of alternating pattern?)

                    • I was wondering if adding a strip of some soft colored lights under the suit under the chin might help define the chin w/o showing too much - could try led's, el wire or even glow sticks (would be nice not to have to find a way to hide a battery pack in a skin tight suit ; - ))
                    • Might also experiment w/ putting a thin strip of white tape in the black areas @ the chin or back tape in the white areas to break up the pattern and create a thin line of one or the other
                    • Unlikely but...Are there any Premiere or AE filters that might be used to sharpen the edges of the suit, including the chin, when processing the footage and make it easier for the tracker to track?
                    • Or...?



                    • 7. Re: Problems tracking face in patterned zentai suit
                      bogiesan Level 4

                      I had to look up what a zentai suit was. This is a costume, right? I'ts not a legitimate motion capture system apparatus. You must provide something for the tracker to follow. LED lights are an interesting suggestion but why can you not have an expert sew some appropirate targets onto the facial area? Rick Gerard's recent adivce was to use a (possibly red) triangle with a (possibly contrasting green) dot in the middle of the triangle. The triangle defines the search area and the dot provides a distinctly unique target within the search area. Three of these (left eye, right eye, mouth) would provide Mocha with planar information to distort into perspective on some shots where the face rotates or tilts.


                      best of luck

                      • 8. Re: Problems tracking face in patterned zentai suit
                        ZentaiKitty Level 1

                        No - It's a costume for the film, not a motion capture suit, nor was it ever indicated it was for motion capture. Those usually have a lot of dots all over them for the motion capturing tools to train on, and make it possible to do modeling on top. This will be footage of the suit w/ a fairl flat cartoon face on top, but no true 3d modeling requirements. (However, we are planning some later shoots trialing using multiple kinects with Blender for 3d motion capture - should be lotsa fun, specially as we tweak the software add ins on the fly... urrgh! ; - ))


                        We won't be modifying the suit in a way that is not possible to change back easily for this shoot.


                        I actually suggested the red tape lines on the face, but Mylenium indicated there were some possible issues with that, particularly w/ planar tracking in Mocha and poses that turn the head all the way down, so sometimes the dots will disappear. So, I'm looking at some possible alternatives for making the facial plane more defined within the footage that would not rely on particular pieces always being visible. (Lighting up the chin seemed like a way to always have contrast b/t chin and chest.) That's why I'm still spitballing around ideas. : - )


                        new one:

                        - maybe make a hard to see trace of the chin on the chest in white or black tape to have that area defined still when chin goes towards chest (not that apt for other shots, but maybe this one deserves some special handling) or look at lights placement closer to those - might even be easier to mount and hide batts in the bra