16 Replies Latest reply on May 5, 2015 4:40 AM by Mathias Moehl

    Mocha & Keylight: problems with a greenscreen on the iPad

    _snickers

      Hey guys,

       

      at the moment I am doing my first greenscreen project with an iPad (and a TV). But I have a few problems with the turbulent footage.

      In the first picture you can see the tracked path on the iPad with Mocha. Everything moves really fine in the final result (see second picture). But there is a lot green left in the person's face and around the screen that I need to get rid of.

       

      So my first question: How do I get rid of those green diffuse edges around the screen and the green in the face?

      I tried several keying effects including keylight etc. – nothing worked. Either the green from the plants around dissappeared or it looked really edged. Is the only possibilty using moving masks? Or is there a more precise method?

       

      Second question: How can I isolate the hand from the greenscreen while interacting "over" the iPad? I have a few sequences where the hand of the person is interacting on the (green)screen. So I need to get the screen behind the hand although everything's moving :-/ Is that possible?

       

      Thanks a lot for your support!

       

       

      Mocha.png

      AE.png

        • 1. Re: Mocha & Keylight: problems with a greenscreen on the iPad
          Rick Gerard Adobe Community Professional & MVP

          First, next time you are tracking tv screens and ipad screens don't put green on the screen. It is entirely unnecessary and makes, as you can tell, your work much harder. You also do not need any tracking marks on any objects that have a clearly defined plane and some detail like the TV and the iPad. A blank screen, or maybe better yet, a medium gray screen will give you the ability to grab reflections off the screen and overlay them on your inserted video for more reality.

           

          Second, if you are using keylight you can suppress the color green by just turning it on. That should get rid of most of the problems.

           

          As a last resort, if you have Red Giant Colorista, you can actually pick and mask specific colors and apply color corrections to just those colors.

           

          You can also use selective color and a bunch of other tools to remove that spill. Then do some color grading on the inserted image so that it looks like it is actually part of the device. It's way too bright and there are no reflections on the screen. You'll have to fake them.

           

          You should also learn about light wrap and try to color match your tracked footage a little better.

          1 person found this helpful
          • 2. Re: Mocha & Keylight: problems with a greenscreen on the iPad
            _snickers Level 1

            Thanks a lot Well I thought a green screen is the best colour to use – but good to know that it isn't. In some tutorials I saw people using it and thought it would work. Well, now I have to work with this footage

             

            In keylight I get rid of most of the green. But I've tried several settings not getting a satisfying result.

             

            I think I will check out "Red Giant Colorista" and some light wrap tutorials and see how it works.

             

            But do you have a better general idea in case of tracking? I mean Mocha is nice but is there maybe a better opportunity, also to get rid of the green parts?

             

            And what about the interacting hand "on the screen"? How can I isolate it?

             

            by the way: the picture above was a first trial, not the final result but I hope to get some tips from you to make the best out of the footage...

            • 3. Re: Mocha & Keylight: problems with a greenscreen on the iPad
              Navarro Parker Level 3

              The main reason for shooting on green is to make it possible for things to pass in front of the green without a lot of frame-by-frame rotoscoping work. So you have made a hole with Keylight. Now put your simulated screen comp *behind* your keyed footage. Now the person's hands will be on top of the screen. If you didn't shoot on green, you would have to manually cut out the person's hand for each frame it overlaps the screen. (This process has been improved by AE's Rotobrush and CC's new Refine Edge tool, but having a good chromakey is the fastest and easiest way to accomplish that task.)

               

               

              To get rid of green reflected on surfaces, traditionally you need a spill suppressor. After Effects ships with one called... Spill Suppressor! Apply that and see if that gets rid of your reflected green on your person. There are other 3rd party suppessors. My favorite is Key Correct's Spill Killer and DVGarage's Conduit Unspill. But those cost money and Conduit has a steep learning curve.

               

              As far as Mocha goes, it's the best software out there for tracking flat surfaces.

              • 4. Re: Mocha & Keylight: problems with a greenscreen on the iPad
                _snickers Level 1

                That's exactely what I want to do, so it isn't that wrong, isn't it? But the main issue is, I can do what I want – if I try to key out the colour of the greenscreens than all the rest of my footage turns into another colour. Quite independent how I change the settings and values of for example "Keylight" or "Spill Suppressor", I've tried almost all effects, but it still changes the whole footage and leaves some green pixels on the rest of the screens I've also tried to use one effect several times. How do you usually do it? Do you use one effect or more?

                • 5. Re: Mocha & Keylight: problems with a greenscreen on the iPad
                  Navarro Parker Level 3

                  Have you tried keying it in Premiere Pro? Its Ultra Keyer is surprisingly good.  Load your footage into Premiere, Key it with Ultra, then import that Premiere Project into AE using Dynamic Link. Any changes you make to the Ultra key, instantly show up in After Effects.

                   

                  Keylight is very powerful, but new users usually end up tweaking too many settings.

                   

                  Keylight Basics

                  To get a really good basic key, there's only a couple things you need to mess with on Keylight: Screen Matte> clip black and clip white.

                   

                  First use the eye dropper to select your green.

                   

                  Switch the view mode to Screen Matte. Adjust Clip Black and Clip White so that your talent is completely white inside and the green background is completely black.  You can also use View>Status for an exaggerated view of this (Black is 100% transparent, white is 100% opaque, and gray areas are translucent. Green areas are also receiving despill correction.)

                   

                  And that's pretty much it to get a really good basic keylight key.

                   

                  If you need more spill supression after that, apply the Spill Supressor effect.  But again, try keying in Premiere Pro and see what you get.

                  • 6. Re: Mocha & Keylight: problems with a greenscreen on the iPad
                    Rick Gerard Adobe Community Professional & MVP

                    Green isn't wrong, but you'll get a better result using tracking and roto using a dark gray screen unless the objects passing in front of the screen are very complex. The dark gray screen allos you to pick up reflections and add them to the composite easily. It just looks more real. Mocha can generate the roto as well as the track. If you have a lot of movement then you have the same problems that you have with a bunch of movement on a green screen set. Edge contamination, motion blur, details being lost. With lots of detail and lots of movement you need a very good camera with a very good lens and a lossless or nearly lossless original. Shooting with a professional cine camera that shoots raw or 10 bit uncompressed is the only way to capture that kind of fine detail and pull it out in a key. Consumer and prosumer cameras just dont give you the color information or the resolution necessary to really do a great job with fine details. Acceptable for most work but not great.

                     

                    I guess my point is that if you shoot on a green screen you have to plan for color suppression and spill control on set. Plan your back lighting, and accept the fact that you'll have to do a bunch of color correcting and color suppression that you don't have to do if you shoot for roto. This gets into art direction, costuming, and set design. It's all about planning and the quality of the photography. If you are going to suppress green then you better not have anything in the scene that is important and green. Look at what happened to the green in the flower arrangement and the green in the plants in the window in my example below. A good set designer would have worked around that or let you know that you needed to do some rough roto work on the piece to restore those greens in the shot.

                     

                    I've done a ton of screen replacements for phones and ipads. We carefully plan the shoot so that there's just a bit of simple roto work. If we do need to use green screen (or blue or orange depending on the art direction of the scene) we keep the exposure levels down and the reflected light down. The biggest problem you have in your scene is that the green from the TV and the green from the tablet are so bright they are acting as light sources for the set. This is where an experienced cinematographer or vfx supervisor would have saved you a bunch of time in post.

                     

                    When you're stuck with footage with a lot of spill you have to isolate that spill and try and remove as much color as you can.

                     

                    Try this with keylight. I'm using your mocha.png. I set keylight to view the corrected source then set foreground color correction to on and set color suppression to green, then I set edge color correction to on and set color suppression to green. There is no green left in the image. Your green edges should be gone completely. As you can see, with these settings, the green from both screen is completely gone.

                     

                    chromaKey.jpg

                    To restore the green in the flower arrangement and the window just duplicate your keyed layer, do a rough roto around the flowers and the plants and reduce the color correction and color suppression to bring back the color there.

                     

                    This shows you that you'll need another layer to fix the color problems. Most of my work that involves keying results in a comp with 5 or 6 layers just to control the color, most of them copies of the main plate (green screen footage) two or three copies of the replacement footage (things that fill the green screen) a layer for light wrap, a layer for lighting effects, one or two layers for reflections if there are shiny things that are being keyed. and an adjustment layer for final color correction. Sometime two or three adjustment layers for color correction.

                     

                    Great compositing is seldom if ever two layers and one instance of Keylight.

                    1 person found this helpful
                    • 7. Re: Mocha & Keylight: problems with a greenscreen on the iPad
                      _snickers Level 1

                      Thanks for your answer I will try keying it in Premiere Pro today. That also seems to be a good idea Thanks for your instruction I know the basics of keylight but I'm still not satisfied with the result

                       

                      I've also tried "Key Correct" at university and it seems to be a really good quality to get rid of the green in the face

                      • 8. Re: Mocha & Keylight: problems with a greenscreen on the iPad
                        _snickers Level 1

                        Wow, what a helpful and detailed answer!! Thank you so much

                         

                        Really interesting how you would do this. Next time I know a lot more about filming on the set. I think this time the only way is using masks. I've tried it yesterday and think it's possible. Hopefully also for the rest of the movie

                         

                        I also got your result by using keylight and tried to experiment with more than one layer. Think it's the only way.

                         

                        Now I will start with the rest of the movie and see how it works

                        • 9. Re: Mocha & Keylight: problems with a greenscreen on the iPad
                          _snickers Level 1

                          One more question after this color stuff:

                           

                          Could you explain me a little bit about this light wrap? How to make the iPad look more realistic? I couldn't find a tutorial for phones / tablets or other devices. In "Key Correct" there is a Light Wrap but if I try to use it, nothing happens

                          • 10. Re: Mocha & Keylight: problems with a greenscreen on the iPad
                            Todd_Kopriva Level 8

                            See this page to learn about light wraps:

                            http://provideocoalition.com/ryoung/story/how_to_light_wrap/

                            1 person found this helpful
                            • 11. Re: Mocha & Keylight: problems with a greenscreen on the iPad
                              _snickers Level 1

                              Thanks for this tutorial. It was really helpful

                               

                              But can someone tell me how I can create some glass reflections on the iPad? It is really difficult

                               

                              I have basically three layers now. The first one with Keylight, the second one "behind" it with my tracked footage and a background.

                               

                              So how can I create a 3D glass effects on the iPad while it is moving? I've tried to experiment with several effects including CC Glass but nothing really changed.

                               

                              Btw: I mean something like shown here (from advertising):
                              http://images.apple.com/euro/support/products/images/ipad-hero.jpg

                              Maybe a bit more realistic?

                              • 12. Re: Mocha & Keylight: problems with a greenscreen on the iPad
                                Rick Gerard Adobe Community Professional & MVP

                                You will need a black and white fairly high contrast image of grandma's face with glasses on taken from the front. If that's not possible try finding one.

                                 

                                Overlay the black and white image on the ipad screen and, if there's any motion on the ipad, then you'll have to animate some motion on the screened photo. Something like this set to be a 3D layer with the blend mode set to screen and opacity adjusted. You'll also probably to warp the image and maybe blur it a bit more.

                                 

                                screen.jpg

                                Gives you something like this:

                                 

                                reflection.jpg

                                This needs a lot of work with scale and distortion and would likely need to be animated, but you get the idea.

                                • 13. Re: Mocha & Keylight: problems with a greenscreen on the iPad
                                  _snickers Level 1

                                  as simple as it is ingenious

                                   

                                  The good thing is, we have a lot of footage filmed from the perspective of the iPad camera so I can use this videos as the reflection.

                                   

                                  Amazing!

                                   

                                  Thank you so much!!

                                  • 14. Re: Mocha & Keylight: problems with a greenscreen on the iPad
                                    Rick Gerard Adobe Community Professional & MVP

                                    As I have said many times before a good composite is never 2 layers. Most of mine are 5 or 6. Many are 10 or more. Glad I could help.

                                    • 15. Re: Mocha & Keylight: problems with a greenscreen on the iPad
                                      _snickers Level 1

                                      Sorry for the misunderstandig I meant basically there are three layers...Of course I do have more Thanks a lot again!

                                      • 16. Re: Mocha & Keylight: problems with a greenscreen on the iPad
                                        Mathias Moehl Adobe Community Professional

                                        I have created these two tutorials that explain in detail how to do this kind of compositing of screen inserts (including reflections):

                                         

                                        screen insert without green screen:

                                        Advanced Screen Insert Corner Pin with mocha and After Effects | mamoworld

                                         

                                        screen insert with green screen (and some nice card dance effec, but the same workflow also works for a normal corner pin):

                                        Card Dance Effect with mocha and After Effects | mamoworld