19 Replies Latest reply on Oct 4, 2011 8:45 AM by Eliminateur

    Changing lip colour

    Eliminateur Level 1

      Hello,

      i'm experiencing the following conundrum and i'm sure someone knows an elegant solution to it:

      I have video which consistts of a closeup of a mouth talking, the video needs to be in B&W(no problems there), but i need to change the colours of the lip from the natural red to fucsia and then to violet.

      Is there a way to do this somewhat "automatically" in AE CS 5.5?, what i thought would be the solution would be to do a "motion mask" over the moving lips(but with care regarding when teeth are shown!, i don't want to colour the teehts, that would look very bad) and then playing with hues and saturation to obtain the desired color.

      So, is it possible for AE to track a "dynamic mask" for this job?, or i need to mask frame by frame?

       

      regards,

      Eliminateur

        • 1. Re: Changing lip colour
          Mylenium Most Valuable Participant

          There is no way to track masks in AE. Any such solutions involve using scripts to apply motion data from Nulls to masks post facto. So if there is no chance to shoot this properly´again , then manually masking it will probably be.

           

          Mylenium

          • 2. Re: Changing lip colour
            Dave LaRonde Level 6

            Eliminateur wrote:

            the video needs to be in B&W (no problems there), but i need to change the colours of the lip from the natural red to fucsia and then to violet.

            Is there a way to do this somewhat "automatically" in AE CS 5.5?

             

            There is: the Keep Color effect -- everything except the designated color turns to B&W.  Combined with either the Change Color or Change To Color effects, both of which can be animated, you should be able to do this with a minimum of fuss.

             

            It may be easier to do this on a duplicated layer, restricting the image size with a feathered garbage mask.

            1 person found this helpful
            • 3. Re: Changing lip colour
              Eliminateur Level 1

              Dave,

              interesting, thing is that the lips are not painted, so it's not a uniform colour at all, how would the keep color fx work in that situation?.

              i've been eyeing some tutorials about removing blemishes (http://www.videocopilot.net/tutorials/blemish_removal/) that i think i can use also combining a feathered garbage mask to restrict it to the immediate areas around the lip

               

              Mylenium, what do you mean by that, the video is already shot properly, there's no other way to shoot it...

              • 4. Re: Changing lip colour
                Dave LaRonde Level 6

                Eliminateur wrote:

                interesting, thing is that the lips are not painted, so it's not a uniform colour at all, how would the keep color fx work in that situation?... what do you mean by that, the video is already shot properly, there's no other way to shoot it...

                It's too bad the lips weren't painted; I think that may have been what Mylenium was talking about.

                 

                However, if there is a sufficient amount of difference between the lip color and face color, you might be able to do it; the inside of the mouth would be a different story.

                • 5. Re: Changing lip colour
                  Eliminateur Level 1

                  i thought of that too, but unfortunately there wheren't any bright green lipsticks at hand .

                  since its a girls lip i think i have enough contrast/color difference to play around.

                  i wonder if i motion track the lips(but which one to track, upper or lower, hmmmm) and then make a mask what would happen, time to do some experiments!

                  • 6. Re: Changing lip colour
                    Fuzzy Barsik Level 4

                    Try Roto Brush Tool to create dynamic mask.

                    Here are not quite contrasted lips

                    AE. Changing Lips Colour 01.jpg

                    Here is the draft output

                    AE. Changing Lips Colour 02.jpg

                    • 7. Re: Changing lip colour
                      Bob Currier Level 3

                      Another approach is to use Color Finesse's secondary color correction. From the sample frames seen above it should be pretty easy to get a secondary qualifier to select just the lips, then you can color correct as desired. Probably a hue adjustment and saturation increase. That would give you a completely automated way to do it.

                       

                      If you can't get the secondary to select just the lips, worst case you'll need to use a garbage mask to roughly select the lip area. Easier than a full roto.

                       

                      Bob Currier

                      Synthetic Aperture

                      • 8. Re: Changing lip colour
                        Eliminateur Level 1

                        Thanks for the pointers guys, i particularly like the roto brush idea, but....

                        the lips have barely any contrast with the surrounding skin AND the video is extremely noisy and grainy, thus roto brush loses lock quite easily, specially in the outer edges of the mouth, another very annoying thing is that it includes the teeth/tonge when the mouth opens.

                        I tried to leave it alone in "auto" to see if it was usable but it's no good(usually loses hald the mouth after 3 frames), i end up needing to adjust frame by frame. .

                        i'll see if i can do something because editing frame by frame plus propagation processing delay is going to take an entire day for a minute of video

                         

                        Bob, i would need to buy color finesse plugin for that which isn't the idea

                        • 9. Re: Changing lip colour
                          bogiesan Level 4

                          Since the oputput of the roto is a mask, you can force a copy of the lip video to have extremem contrast, enough to process the roto. Many of AE's cooler applications involve circuitous application of many filters that create movies or precomps that are used to control or provide input to other filters.

                           

                          bogiesan

                          • 10. Re: Changing lip colour
                            Eliminateur Level 1

                            hmmm but rotobrush works on the source, not the comp so it only takes the original footage IIRC(so i can't apply effects)...

                            btw, even with full contrast, the lips are lost on the surrounding skin.

                             

                            in any event, this does not solve the problem of having to manually exclude the teeth when the mouth opens, so i'm going to end up rotoscopping the rotobrush frame by frame anyway

                            • 11. Re: Changing lip colour
                              Fuzzy Barsik Level 4

                              What David said is it's not forbidden to e.g. render out an intermediate with increased contrast or whatever and then reimport it into your composition to help Roto Brush Tool find edges.

                               

                              It doesn't obviously solve the issue with opening and shutting mouth. Solving this one highly depends on the 'event' frequency, blending with tongue complexity etc. etc. However, bear in mind, you may set up Roto Brush keyframes at any point you like and propagate them both forward and backward.

                               

                              BTW, Color Finess is included with After Effects...

                              • 12. Re: Changing lip colour
                                bogiesan Level 4

                                Thank you, fuzzy.

                                Back in the olden days of AE2, we learned to pull excellent chromakeys off terrible footage by exporting copies fo the footage that had been really cranked in a variety of bizarre filters.

                                 

                                bogiesan

                                • 13. Re: Changing lip colour
                                  Eliminateur Level 1

                                  about event frequency, it's continuous talk, so it varies A LOT every frame, non-stop.

                                  more than the tongue the problem are the teeth

                                   

                                  i'm still trying to grasp the roto brush properties, about keyframes and propagation(like, to setup keyframe do i need to lower the forward limit and then brush a new area?, what's the difference between propagating only forward or propagatring forward and backward from X point?, and what kind of frame is better to use as RB keyframe....

                                   

                                  hmmm i hadn't tought of doing another comp only for the main video to play with levels and then render it..... (arggh the tought of having to warp stabilize that again.... time flies awayyyy...).

                                   

                                  letsay i manage to pull off a combo to get best rotobrushing, i have the retouched clip with the roto mask on, how do i split only the mask from that layer(since inside the mask the content it's all retouched) as roto mask does not generate a normal adjustable mask that i can apply to an adjustment layer or copypaste?

                                  • 14. Re: Changing lip colour
                                    Fuzzy Barsik Level 4

                                    Hm-m-m, from where I'm standing teeth should be no problem at all, but rather areas with hairs or tongue overlapping lips, e.g.

                                    AE. Speaking Mouth Roto.jpg

                                    The difference between propagation approaches is in the following.

                                    Roto Brush Tool is unable to identify gap between lips you need to exclude from mask when mouth is shut, and it doesn't know what should eventually be excluded when mouth just starts opening. Extreme opened mouth position gives much more understanding, so it's better to set up keyframe here.

                                     

                                    See more help about Roto Brush Tool.

                                     

                                    Regard to using rendered intermediate for further processing, you use it for creating Alpha Matte, e.g.

                                    AE. Speaking Mouth Roto 02.jpg

                                     

                                    See more help about Track Mattes.

                                    See also help section about compositing.

                                    1 person found this helpful
                                    • 15. Re: Changing lip colour
                                      Eliminateur Level 1

                                      letme see if i follow your last step: so you leave the top layer "intact", dupe it, rotobrush the dupe and then track matte(from what i can see, alpha matte?) it to the top layer?

                                      i'm assuming you then play with (threshold, hue, luma) to make it white and use it as aan alpha matte applied to an adjustment layer?

                                      isn't it easier to just rotobrush the top layer and leave it in normal mode?

                                      (my top layer is the footage desaturated for a B&W effect over the painted lips).

                                      • 16. Re: Changing lip colour
                                        Fuzzy Barsik Level 4

                                        No, I posted the latter screenshot just for demonstration purposes of how to create Alpha Matte out of something.

                                        You should do something like the following:

                                        1. Increase the contrast or whatever of your original footage that may help Roto Brush Tool find lips edges.

                                        2. Render intermediate out of that comp, reimport it, place on top of all the rest layers and call e.g. 'Matte for Lips'.

                                        3. Apply Roto Brush Tool onto 'Matte for Lips' layer.

                                        4. Duplicate your 'intact' original footage.

                                        5. Desaturate the lower one and call it e.g. 'Main View', 'B&W View' or whatever you like.

                                        6. Call the layer in between e.g. 'Coloured Lips', animate Hue and set 'Matte for Lips' layer as Alpha Matte.

                                        7. Play with blending modes to get better look...

                                        • 17. Re: Changing lip colour
                                          Eliminateur Level 1

                                          turns out i don't need to render intermediate, if i simply put the roto brush last in the effects chain i can do it in "realtime".

                                          i'm playing with some levels and i'm getting some hopeful results(what's funny is that it *looks* horrible to the eye but rotobrush seems to like it, if i try to use remove grain before it, it loses lock in some parts), it's only losing lock in the borders, but it can differentiate lips/teeth/mouth with mild success.

                                          footage.jpg

                                           

                                          btw: i think i found a bug/oddity that i can't explain due to settings and whatnot with using rotobrush and warp stabilizer, i haven't been able to test it on a clean comp yet, but i think that to reproduce it(based on my current comp):

                                          load some footage, scale it so that it's larger than comp size and move it offcenter as well.

                                          Warp stabilize it with stabilize it with no motion/position/stabilize only

                                          then select roto brush and when you click to mask something, the layer content JUMPS upleft, the moment you release the click, the correct area is marked.

                                          IF you try to undo it, it MOVES the footage in the composition way off your origianl set position, you need to undo again to go back(so, two undos for a single click of roto brush)

                                          • 18. Re: Changing lip colour
                                            Rick Gerard Adobe Community Professional & MVP

                                            Judging from your screenshot you are facing a terrible uphill battle. The blacks are curshed and you've got a bunch of large areas around the mouth that have exactly the same pixel values as the lips. Unless your actress has a skin condition there's no way the image represents what she actually looks like. This footage is just not going to work very well.

                                             

                                            When planning something like this it's critical to get the best video you can. This video looks way under exposed, highly compressed, and, unless it's you that's jacked the color around, the white balance is way off. To get any kind of acceptable separation you're going to need to have pixel values that are at least a few points apart. You can see the values in the Info Palette. With the image you've provided in your screenshot the only way to get a clean cutout of the lips is to do it manually. RotoBrush won't even make a good selection of just that frame. 

                                             

                                            Do you have a frame or two of the unprocessed footage? Maybe one of us can come up with a solution if we see the original footage.

                                             

                                            If there's any way to get the girl back in front of a camera I'd reshoot. It will take much less time than it will take monkey around with what looks to be horrible footage.

                                            • 19. Re: Changing lip colour
                                              Eliminateur Level 1

                                              Rick,

                                              indeed that was the capture taken after toying with some filters to see if it would lock better.

                                              And yes, i think it's an uphill battle that i'm not going to win, i guess next time i'll need to take better footage(with a good cammera if i can get one) under better lighting AND with some kind of color diferentiator(lipstick/whatever).

                                              I'd reshoot it if i could, but as usual with these kind of projects, i've ran out of time(but i've learned a huge deal)

                                              these are a couple raw frames from the footage( and this are ones that look best...):

                                              repeat.MPG_snapshot_00.02_[2011.10.04_12.15.06].jpg

                                              repeat.MPG_snapshot_00.08_[2011.10.04_12.15.32].jpg