12 Replies Latest reply on Oct 6, 2010 12:58 PM by Kevin Goldsmith

    Turning a Photoshop action into a pixel bender script

    roy_martin Level 1

      We have a photoshop action that takes images and 'plasticifies' them. Specifically the action performs several filters that we're looking to convert into pixel bender scripts so that the action can be performed in Flash. I'm trying to discover if there are existing examples to perform these actions, and if not if it would be possible to create them. The filters that are being used are:

       

      • surface blur
      • diffuse
      • layer copy
        • high pass filter
        • layer overlay
      • hue / saturation
      • shadow / highlight
      • unsharp mask

       

      I've attached the script to the post for reference and provided an example below. If this isn't something that we could combine several pre-existing filters together, would someone be up for working with me to create this or provide some guidance on how to move forward?

       

      Original Image
      angelina_jolie_original.jpg
      Plasticify Effect
      angelina_jolie_plasticify.png

       

       

       

      File: autoPlastic_test.atn (created in Photoshop CS4).

      URL: http://drop.io/t7gez0g

       

      Thanks,

      Roy

        • 1. Re: Turning a Photoshop action into a pixel bender script
          Royi A Level 1

          It is all can be done easily in PB except for the Surface Blur.

          I don't know which algorithm Adobe uses for Surface Blur.

          If you know which one I could assist you more (Or any Adobe representative would just give the type of algorithm).

           

          Some of Edge Preserving Blur algorithms, Such as Anisotropic Blur requires some math I don't think is easy to implement in PB at his current state.

           

          Hopefully in the near future more flexibility will be added to apply iterative algorithms and other more advanced tools (Such as Conjugate Gradients methods).

           

          I guess more demand from the community will make those features first priority.

          • 2. Re: Turning a Photoshop action into a pixel bender script
            unique_screenname_here Level 3

            As an aside, the Pixel Bender plugin can be scripted via Photoshop's actions so you can actually use the Pixel Bender plugin in combination with other Photoshop filters to experiment with converting the filters from Photoshop-native filters to Pixel Bender filters.

            • 3. Re: Turning a Photoshop action into a pixel bender script
              Royi A Level 1

              I haven't checked with version 2.1.

              Yet in version 2.0 it crashed each time when I tried scripting it.

               

              By the way, could you tell us what kind of filter is Surface Blur?

              Is it Anisotropic Blur, Bilateral, Mean Shift?

               

              Thanks.

              • 4. Re: Turning a Photoshop action into a pixel bender script
                roy_martin Level 1

                There are various names for this blur, but the algorithm that we found that gave us the correct solution was one called the smart blur filter, or as GIMP calls it smart gaussian filter. This solution actually documents the algorithm that is necessary through steps in Java, in fact it does such a good job that we can currently use this filter on it's own to get the desired result.This might be a great resource in general for converting filter algorithms into pixel bender scripts.

                 

                Would it be possible to use the logic from this filter to create a pixel bender script? If so, would someone be up for the challange? Here is the information on the algorithm:

                 

                Filter Description

                http://www.jhlabs.com/ip/filters/SmartBlurFilter.html


                Filter Algorithm

                http://tblurb.com/29U0Uw

                 

                All Filter source files:

                http://www.jhlabs.com/ip/filters/Filters.zip

                • 5. Re: Turning a Photoshop action into a pixel bender script
                  Royi A Level 1

                  This filter is strange.

                  It doesn't have a normalization factor (At least I couldn't see it, I'm not a Java skilled programmer), Which might change the the Intensity of the image.

                  Yet using a Threshold for blurring is like a primitive Bilateral Blur Filter, Which works pretty slow in PB (Though theoretically could be implement in a very fast manner on the GPU).

                  I'll try to contact the author to get a better understanding of the filter.

                   

                  Morover, in order to speed it up, If implemented on the CPU I would only use it on the Luminosity Channel (In LAB or at least YUV). I am not sure it will make any difference on the GPU.

                  I think in the GPU each basic variable (Pixel) is a vector of 4. Applying something on 4 of items in the vector or only 1 would "cost" the same.

                  Someone of the AIF could confirm it?

                  • 6. Re: Turning a Photoshop action into a pixel bender script
                    Kevin Goldsmith Level 3

                    The Bilateral Blur in After Effects is implemented in Pixel Bender. It  was tricky to get the best possible performance for it. Pixel Bender  works pretty well for Anistropic filtering as well, we are doing that  now for some new stuff.

                    Royi A wrote:

                     

                    I think in the GPU each basic variable (Pixel) is a vector of 4. Applying something on 4 of items in the vector or only 1 would "cost" the same.

                    Someone of the AIF could confirm it?

                    This is sort of true. We do take advantage of the vector types to generate SSE code, but we do it not only within a pixel, but also across multiple pixels so a pixel1 value within your kernel may be computed with pixel1 values from neighboring pixels to improve performance. In general, it is best to write your code naturally and then only look to optimizing at this level if performance is unacceptable and the more obvious problems (minimizing conditional branching, for example) do not solve your problem.

                     

                        Kevin

                    • 7. Re: Turning a Photoshop action into a pixel bender script
                      unique_screenname_here Level 3

                      Royi A wrote:

                       

                      ...

                      Yet using a Threshold for blurring is like a primitive Bilateral Blur Filter, Which works pretty slow in PB (Though theoretically could be implement in a very fast manner on the GPU).

                      ...

                      In comparison to what? I take it you're running the Bilateral Blur filter that ships with AfterEffects?

                      • 8. Re: Turning a Photoshop action into a pixel bender script
                        Royi A Level 1

                        AIF Kevin wrote:

                         

                        The Bilateral Blur in After Effects is implemented in Pixel Bender. It  was tricky to get the best possible performance for it. Pixel Bender  works pretty well for Anistropic filtering as well, we are doing that  now for some new stuff.

                        Royi A wrote:

                         

                        I think in the GPU each basic variable (Pixel) is a vector of 4. Applying something on 4 of items in the vector or only 1 would "cost" the same.

                        Someone of the AIF could confirm it?

                        This is sort of true. We do take advantage of the vector types to generate SSE code, but we do it not only within a pixel, but also across multiple pixels so a pixel1 value within your kernel may be computed with pixel1 values from neighboring pixels to improve performance. In general, it is best to write your code naturally and then only look to optimizing at this level if performance is unacceptable and the more obvious problems (minimizing conditional branching, for example) do not solve your problem.

                         

                            Kevin

                        Kevin, how do you iterate in the Anisotropic Filter?

                        One the things I miss is to loop a graph or something like that to iterate over the image.

                         

                        Regarding the Bilateral Filter, My first PB kernel was a Gaussian Blur, just to get the feeling.

                        At first I didn't use its separability property.

                        It was slow, very slow!

                        For the Bilateral Filter you have to do even more operations. Unless you implemented its "Separable Bilateral" approximation, Or any other known optimization (Grids, Linear Approximation, etc...).

                         

                        What about the GPU case regarding what I wrote?

                        Let's set I convert the image into LAB space and perform Gaussian Blur and each time only operate one CurrentPixel.r (The first item in the "Pixel" vector), will that make any performance difference?

                        Or even make it completely different. Creating a node which take as input image4 and output of image1 (L Channel). Then Gaussian Blur (2 Nodes) on image1, would that be faster (Again, GPU wise)?

                         

                        Thanks.

                        • 9. Re: Turning a Photoshop action into a pixel bender script
                          Royi A Level 1

                          Could anyone address that (The Above Message)?

                           

                          1. Will applying something into only once item in a pixel vector speeds up thing when executed on the GPU?

                          2. How could you set the iteration number when using Anisotropic Diffusion? Or did you set the iteration number before and just iterate the same kernel using graphs?

                          3. Could you shed some light about the Bilateral Filter. The Separable Approximation creates artifacts for large Radius. Using Bilateral Grid doesn't work well for small Radius. The O(1) methods are adjacent Pixels dependent hence can't be used.

                           

                           

                          Thank You.

                          • 10. Re: Turning a Photoshop action into a pixel bender script
                            Kevin Goldsmith Level 3

                            I can't go too deep into the details of our implementation.

                             

                            I can say that you point out correctly that we can't do the iteration of the Anistropic filter in Pixel Bender. That is currently handled in C++. Each time we develop new features using Pixel Bender, we find out what doesn't work well for practical development and we use that to help us figure out what we need to add to make it work better for everyone.

                             

                            For Bilateral, we use different kernels for different parameter values.

                             

                            Royi A wrote:

                             


                            What about the GPU case regarding what I wrote?

                            Let's set I convert the image into LAB space and perform Gaussian Blur and each time only operate one CurrentPixel.r (The first item in the "Pixel" vector), will that make any performance difference?

                            Or even make it completely different. Creating a node which take as input image4 and output of image1 (L Channel). Then Gaussian Blur (2 Nodes) on image1, would that be faster (Again, GPU wise)?

                             

                            My feeling is that it shouldn't make a significant difference, but if I had time and the performance of the first approach wasn't where I wanted it, I would try some experiments with the second.

                             

                                 Kevin

                            • 11. Re: Turning a Photoshop action into a pixel bender script
                              Royi A Level 1

                              I see.

                              Well, One day hopefully we'll be able to Loop over nodes.

                              do you think it is possible or is it a limitation built in the GPU and we'll never see it in PB?

                              I'm optimistic since manually I can do that so it is a management thing :-).

                               

                              Thanks for sharing the info.

                              • 12. Re: Turning a Photoshop action into a pixel bender script
                                Kevin Goldsmith Level 3

                                It is a Pixel Bender graph limitation, not a GPU one. (which means we can fix it).