17 Replies Latest reply on Sep 26, 2011 11:02 AM by AIF Bob

    Pixel Bender Wishlist

    Christian Lett Level 3

      I wanted to submit some Pixel Bender feature requests via the official Adobe feature request form, but couldn't find Pixel Bender in the list of products.

       

      So in the mean time, maybe we can put our PB feature requests here.

       

      Here are a few to start:

       

      - Random number functions - e.g. float[2,3,4] random([seed]) or float[2,3,4] random([seed], min, max). This is pretty basic programming language functionality that would help enormously when developing noise or fractal filters

       

      - After Effects parameter groups - I develop AE plugins using PB, and it would be useful to have a <parameterGroup> tag that we could use to organise parameters. I guess it would wrap parameters like so:

      (kernels)

      <parameterGroup name="myGroup" displayName="My Group">

           parameter float myParam < ... >;

      </parameterGroup>

       

      (graphs)

      <parameterGroup name="myGroup" displayName="My Group">

           <parameter type="float" name="myParam">...</parameter>

      </parameterGroup>

       

      - Being able to know the dimensions of an input image within evaluatePixel(); This would be useful for implementing repeat edge pixels, for example

       

      - Ability to give alternate input sources user-friendly names. Currently if you add a second input source to a PB kernel or graph, in After Effects this simply appears labelled as "Input 2". Using an aeDisplayName tag as for regular parameters would fix this.

       

      - In the Pixel Bender Toolkit, why do boolean parameters (checkboxes) appear at the top of the stack, rather than in their coded position?

       

      - Simulated AE parameters for any parameters with "ae..." metadata - e.g. colour pickers, angle controls, dropdowns (int params with popup strings).

        • 1. Re: Pixel Bender Wishlist
          Felix Dames Level 1

          +1 for every feature Christian Lett suggested!

           

          My feature requests:

           

          1.

          Try to speed up the sampleNearest or sampleLinear function!

          If you develop a filter which needs to know pixel information about surrounding pixels these filters (for example blur filters) can get really slow (depends on how often the sample function get called in a for loop) I don't know why this function is so slow but maybe it would be faster if there would be a possibility to sample once every pixel for the whole frame/image (not again and again for every pixel + the surrounding ones) and store this information in a two dimensional array (2D arrays does not exist at the moment) which get stored in the video card memory for fast access and when surrounding pixel information is needed you would not use the normal sample function but instead request the values from the 2D array. Maybe that would be faster.

           

          Another possibility would be to find a way to cache allready sampled pixels so that pixels would not get sampled multiple times.

          Like modern fastblur algorithms work (a 2 dimensional array which get shifted and filled up with new sample values)

          But I don't know how this could be achieved by a programming model like pixelbender where every pixel is calculated in parallel.

           

          I hope you understand what I mean.

           

          2a.

          The possibility to hide or deactivate parameters depending on code. So for example if you choose B instead of A in a dropdown list some parameters get deactivated and could not get changed anymore and greyout/hide because these parameters does not make sense in the setting of B.

           

          2b.

          The possibility to overwrite parameters values. Now it's not possible to change a parameters value by coding.

           

          3a.

          After Effects should integrate pixel bender in form of a plugin so you could update pixelbender inside after effects without waiting for a new version of after effects.

           

          3b. for the future please try to syncronize all updates. This means if you update the pixelbender toolkit and the language there should be an update for photoshop & after effects plugin as well, at the same time and with support for all new language features.

          • 2. Re: Pixel Bender Wishlist
            Christian Lett Level 3

            Your request to be able to set parameters as visible (or not), or just inactive would be an acceptable workaround to calculating the image size for AE plugins.

             

            I'm having to implement repeat edge pixels by having a float2 AEPoint parameter, and using aePointRelativeDefaultValue of [1.0, 1.0]. This is fine, but it's user editable in AE, which could mean incorrect results. By having a new metadata tag <metadata name="aeActive" type="bool" value="false"> or similar for aeVisible, we could set the parameter without fear of it ever being changed by the user.

            • 3. Re: Pixel Bender Wishlist
              unique_screenname_here Level 3

              Thanks. We'll keep these in mind. Keep them coming.

              • 4. Re: Pixel Bender Wishlist
                Felix Dames Level 1

                some more feature requests...

                 

                1.

                GPU support for After Effects (already Todd Kopriva mentioned this in an other After Effects thread as a good feature request)

                 

                2.

                In kernel descriptions the possibility to set an html link for directing users to a html product page or html help page

                and the possibility for a line break and a paragraph to organize multiple rows of text lines in the kernel description value/parameter.

                maybe like this:

                 

                kernel xyz
                <  

                    description : "<p>This is my wonderful xyz filter</p>here is a link to the product page<br><a href='http://www.xyz.com'>xyz Product Page</a>";
                >

                 

                3.

                This is a windows 7 specific one:

                some information first:

                Win7 (maybe vista also) have a new feature for all system drivers. If a driver does not respond for some time windows assumes that there is a problem and resets the driver.This is actually a good feature but a problem for pixelbender!

                If calculations with pixelbender take to long time and the calculation load on the graphics driver is really high then windows 7 recognize that the graphics driver is not responding anymore and resets it. This results in a total crash of the running application which pixelbender was used in.

                It only happens when pixelbender renders in GPU mode so at the moment after effects (only CPU mode) is not affected but the pixelbender toolkit and PhotoShop are affected and crash! The Pixelbender Toolkit 2.5.1 already got a fix for this issue and compensate for that (Thank you for adding that save feature) but for PhotoShop everything is lost.

                 

                But anyway this kind of crash should not happen at all!

                for further reading on this issue I refer to this thread:

                http://forums.adobe.com/thread/746407?tstart=30

                 

                 

                So my feature request for this problem is:

                would it be possible to keep some kind of headroom for the cuda cores or to not use all of them to keep this headroom? so that windows7 would not recognize the driver as not responding anymore? Or in other words: don't pull every calculation power from the graphic card but only 95% of it so that these 5% headroom would be left for windows normal graphics display system?

                An other possibility would be: if calculation time takes longer then for example 1 sec then pause calculation for a short time so windows recognize the driver again and continue the calculation for the next 1 sec and so on and on until full calculation is finished.

                 

                I don't know if that is ever possible but I know other applications where very long (over hours) calculation times on the gpu are possible and they don't crash/reset the windows graphics driver (for example: Mental Images IRAY renderer)

                 

                 

                4.

                Pixelbender Toolkit support for importing movie files and the possibility to sample pixel information from different frames in time of that movie.

                 

                5.

                Import support for more image formats like .tga .psd .tiff .exr (open EXR with access to all layers)

                 

                6.

                In Pixelbender Toolkit an Output window where it is possible to expose values of variables.

                A visual representation of the value of a variable

                Would be great for debugging filters.

                maybe like this:

                PrintToOutputWindow("some text as a string:", [Variable]);

                 

                7. General Pixelbender support for Adobe PremierePro! Not sure if it is correct but I read something about it in an other thread:

                Pixelbender breaks Adobe DynamicLink between After Effects and PremierePro when importing an AE Composition into PremierePro when Pixelbender was used in AE... I'm really not sure about this and haven't tested it myself. But a general support for PremierePro would be great in my opinion.

                 

                8.

                Anti Piracy / Copy Protection System for pixelbender files:

                maybe something really straight forward with serial numbers and all that stuff...

                But at least something to encrypt the final product in some sort of binary file format so that the source code is not readable anymore with a simple text editor!

                I know everything is decryptable but when it's not so easy to decrypt then for the masses of people you have a working copy protection.

                Better this then nothing.

                 

                 

                ups, sorry for the long post... but it just brainstormed out of my mind....

                • 5. Re: Pixel Bender Wishlist
                  Christian Lett Level 3

                  * The option to view a checkerboard (user-defined colours) for transparency like in Photoshop and AE

                   

                  * An info panel for displaying, amongst other things:

                      - the x, y coordinates of the mouse pointer over the image

                      - the RGBA and/or HSV values of the pixel currently under the mouse pointer as 0-1 or 0-255 values

                   

                  * Introduce C-style "switch case" syntax for multiple-choice, e.g.

                   

                   

                  switch (myMode)
                  {
                       case 1:
                            // do something
                            break;
                       case 2:
                            // do something else
                            break;
                       default:
                            // if all else fails...
                            break;
                  }
                  
                  * This may have already been suggested, but the ability to import floating point images would be very useful
                  * Ability to set certain AE parameters as non-keyframeable, perhaps through a "behavior" metadata tag
                  * Allow graph nodes to loop iteratively based on parameter values - for example if writing a blur, the same node could be called n-times depending on how big the blur radius is, instead of sampling huge amounts of pixels in one node call and getting a very boxy blur.

                  • 6. Re: Pixel Bender Wishlist
                    klsteven-qW1idA Level 1

                    My wishlist as a Photoshop user would be, that Pixel bender would finally be a finished Photoshop plugin and not just a beta or whatever you call it from the labs. I remember John Nack as he showed PB at PS World before the release of CS4. Now we are at CS5 and it`s still a beta. So what would we need to have a finished product? I tell you some wishes:

                    1.It should work with every file size, not just with pics, that fit into the VRAM, even if it`s slower in thoses cases.

                    1a. PB must support colormanagement, it does not right now.

                    2.We need filters that are really useful. Most of the current ones or more or less just gimmics for Photoshop use.

                    3.Onscreen tools like the adjustment layers in PS have.

                    4.Navigation like in PS with birdseye view, flick panning rotating etc., etc.

                    5. Basic tools like hand, zooomtool etc. . Right now we only have shortcuts, which not everyone knows. And panning around with a zoomed in picture causes screen artifacting.

                    6.Filter stacking like in PS`s filter gallery.

                    7. Filters that can create a "look" easily.

                    8. Storable filter settings

                    etc.,etc.

                     

                    Come on, make it a real plugin. It has so much potential and the speed is incredible!

                    • 7. Re: Pixel Bender Wishlist
                      Christian Lett Level 3

                      Will PB ever support writing to arbitrary pixels via a separate function to evaluatePixel()? In other words, letting us choose which are the target pixels rather than giving us the coordinate.

                       

                      PB is great for many things, but painful for other applications. Recently I thought about generating a vectorscope as an overlay. However this proved extremely inefficient because, for every output coordinate I'd have to calculate the hue/sat value, then scan through every pixel in my input image building up a value every time I got a match. As you can imagine this is sloooow, even when the input image has been scaled down considerably, and with PBs multithreading.

                       

                      Tackled the other way, all I'd need do is scan through my input image once, calculate the HSL for each pixel which would give me my output coordinate, then build up the value of the relevant pixel in the output image. If my vectorscope had 300px, and my input image is 300x200px, then currently it's 300x300x200 (18 million!) samples, as opposed to just 1x300x200 (60k) if we could choose.

                       

                      Obviously this new function (drawOutput()?) could not be used alongside an evaluatePixel() function, and I guess it would not be able to exploit multithreading but that's a small price to pay for more efficient processing.

                       

                      Would be interested in hearing Adobe's thoughts on this (probably - use the SDK and write proper plugins! )

                      • 8. Re: Pixel Bender Wishlist
                        AIF Bob Level 3

                        Christian Lett wrote:

                         

                        Will PB ever support writing to arbitrary pixels via a separate function to evaluatePixel()? In other words, letting us choose which are the target pixels rather than giving us the coordinate.

                         

                        Not in its current form. The existing programming model for PB was very deliberately chosen to match the programming model that's available in OpenGL. It's known as a gather model - many inputs but only one output for each invocation of the kernel. Writing to arbitrary pixels is a scatter model and that is much more difficult to program without errors.

                        Obviously this new function (drawOutput()?) could not be used alongside an evaluatePixel() function, and I guess it would not be able to exploit multithreading but that's a small price to pay for more efficient processing.

                         

                        Would be interested in hearing Adobe's thoughts on this (probably - use the SDK and write proper plugins! )


                        There are some algorithms that are difficult (or inefficient) to fit into the gather programming model. For several years at SIGGRAPH it seemed as though every paper finshed with a section called "how to fit this algorithm onto the GPU". Some of the hoops the authors had to jump through were quite impressive. Efficient parallel programming is non trivial. With PB we make some common and useful cases easy to write, but we don't cover every possibility.

                         

                        Bob

                        • 9. Re: Pixel Bender Wishlist
                          Royi A Level 1

                          Some notes of mine:

                          • Some info about the image should be transfered into the PB code (In PS and AE). Dimensions, Histogram, etc...
                          • Let us apply kernel using data from many frames in AE, meaning applying temporal algorithms on video streams.
                          • Drop support for "Old" GPU's. You enforce too much limitations on PB. DX10 and up cards sounds great.
                          • Better support for arrays.
                          • Ability to create "Temporay Image" with arbitrary size as a replacement for 2D array. This will let the developers utilize optimizations methods in Algorithms.

                           

                          Some more of my notes are written here:

                          http://forums.adobe.com/thread/716847

                           

                          Thank You.

                          • 10. Re: Pixel Bender Wishlist
                            Royi A Level 1

                            What about new version of PB?

                            Is the project dead?

                             

                            We have new CPU's with AVX support which should improve performance in this regard.

                            Any news about that?

                             

                            Thanks.

                            • 11. Re: Pixel Bender Wishlist
                              AIF Bob Level 3

                              Royi A wrote:

                               

                              What about new version of PB?

                              Is the project dead?

                               

                              We have new CPU's with AVX support which should improve performance in this regard.

                              Any news about that?

                               

                              Thanks.

                               

                              Pixel Bender is certainly not dead. We're continuing to work with Photoshop, After Effects and Flash to make sure that AIF and Pixel Bender support what they and their customers need.

                               

                              As I've written before, the hardware we support is an ongoing discussion within Adobe and we're always trying to balance new features against supporting older hardware. It's not an easy balance to get right.

                               

                              Bob

                              • 12. Re: Pixel Bender Wishlist
                                Royi A Level 1

                                AIF Bob wrote:

                                 

                                Royi A wrote:

                                 

                                What about new version of PB?

                                Is the project dead?

                                 

                                We have new CPU's with AVX support which should improve performance in this regard.

                                Any news about that?

                                 

                                Thanks.

                                 

                                Pixel Bender is certainly not dead. We're continuing to work with Photoshop, After Effects and Flash to make sure that AIF and Pixel Bender support what they and their customers need.

                                 

                                As I've written before, the hardware we support is an ongoing discussion within Adobe and we're always trying to balance new features against supporting older hardware. It's not an easy balance to get right.

                                 

                                Bob

                                Hi Bob,

                                Yo say you continue to work with...

                                Yet there are many reported bugs related to the plug in and none has been taken care of.

                                • 13. Re: Pixel Bender Wishlist
                                  Felix Dames Level 1

                                  1.

                                  It would be very helpful (in Pixelbender toolkit) if the code window and or the image preview window could be dropped off the main application window to place them on a second monitor.

                                  I really would like to have more space for the code! As it is now I often have to scale/move my code window up and down.... this is annoying.

                                   

                                  2.

                                  It would be nice to have access to the After Effects Composition Camera values like:

                                   

                                  Camera Transform:

                                  Point of Interest as a float3 value (xyz)

                                  Position as a float3 value (xyz)

                                  Orientation as a float3 value (xyz)

                                  X Rotation as a float1 value (x)

                                  Y Rotation as a float1 value (y)

                                  Z Rotation as a float1 value (z)

                                   

                                  Camera Options:

                                  Zoom as a float1 value

                                  Depth of Field as a boolean value (on/off)

                                  Focal Distance as a float1 value

                                  Aperture as a float1 value

                                  Blur Level as a float1 value (%)

                                   

                                  these are the values for AE CS5 in AE CS5.5 there are some more because the camera got an update regarding depth of field...

                                   

                                  By getting access to these values it would be possible to write pixelbender filters which work in 3D space and always renders the right perspective of the AE camera.

                                  • 14. Re: Pixel Bender Wishlist
                                    richard7854 Level 1

                                    Pixel Bender support on iOS would be really usefull, it could provide really interesting mobile applications. Also GPU support for flash is really needed feature.

                                    • 15. Re: Pixel Bender Wishlist
                                      chauffeurdevan Level 2

                                      Hi,

                                       

                                      I already sent as a feature request a few months ago something similar to that. What I suggested is to getting the info thru the aeUIcontrol metadata for parameters, for example :

                                       

                                      parameter float4x4 camera_matrix
                                      <

                                           defaultValue: float4x4(1.0, 0. ............................................);
                                           aeUIControl: "activeCameraMatrix"; //AE-specific metadata
                                      >;

                                       

                                      So if you use it in after effects, the connections would be automaticaly made (and the parameter is hidden), in flash, you'll be able to send a 4x4 matrix.

                                       

                                      Beside the cameraMatrix, the zoom, aperture, filmSize, and focal lenght would be useful. Also the current layer transformMatrix should also be sent the same way.

                                       

                                       

                                      Jocelyn Tremblay

                                      • 16. Re: Pixel Bender Wishlist
                                        Royi A Level 1

                                        Any updates on the project?

                                        • 17. Re: Pixel Bender Wishlist
                                          AIF Bob Level 3

                                          There is nothing specific I can say. The AIF team continues to work with Flash, Photoshop and After Effects and we're tied in to their release and announcemnt schedules.

                                           

                                          Bob