I've been using Pixel Bender for over a month now.
Implementing some Image Processing Algorithms into it (Advanced Sharpening, Blurring, NR etc...).
I have some notes regarding it to my special usage - Implementing Algorithms for Photoshop / After Effects.
I'm not a programmer so I thought this would be the right tool for me, just taking care of the Algorithm.
I don't have a deep knowledge of what is possible using GPU's and what's not, All written here is in the spirit of the limitations you made clear in the PDF's of PB. Anyhow, Looking on the supported GPU's by you it seems you support pretty old GPU's. Since it's an experimental project for now and the big changes GPU had in the last 2 years you might want to reconsider dropping support for anything below DX10 to give PB all the features and flexibility it should have.
1. Hidden Codes - I know it is possible (Oil Painting Plug In). This should be a first priority. If I got is write, PB Kernels for Flash are compiled into some kind of a binary form. Let's use that, Let the Photoshop / AE Plug In read those files.
2. CPU Precomputing - There are some computations which suit the CPU much better than the GPU. Could the "Calculate Dependencies" concept be expanded? Let's say compute some (Small) data which later be used by Pixel Bender. A simple case would be Face Recognition. Let's say I could create a Gray Mask (Faces would be brighter grays the rest black...) on the CPU and then this mask would be passed as a source image to pixel bender. The CPU procedure could be created using AS3 or something. I know it is easy to do in Flash, yet what about Photoshop / After Effects?
3. Static Node / Loop of Nodes- Let's say I convert the image into LAB Colorspace. Now I apply an Algorithm, Convert it back into RGB and that's it. Let's say the user have 3 sliders controlling some parameters of the Algorithm applied on the LAB image. Now each time the user moves a slider the whole process is being calculated. The solution should be letting us define "Static Nodes", Which means they are calculated only once. Their result is saved for later use. They won't be regenerated. In the example above, The node which Convert the image into LAB would calculated once and that's it. This is a simple example, but there are decompositions of images which are very demanding and needed to be calculated only once. A generalization of this would be Looping over nodes. Which means do this node till something happens. This would make iterative algorithms possible in PB.
4. Changing Graphs Connections on The Fly - In a later note I'll ask for an UI improvements - Radio Buttons. Let's say I built a Graph with 2 different Local Contrast Algorithms. I allow the user to chose one of them (Radio Buttons). I want to change the graph connections according to his choice. The way things now I have to calculate both and use a boolean variable to chose one of them.
5. Performance Measure - We need some way to measure if the optimizations we do really improves performances. I know there's an option to see the FPS. I don't how accurate this is. Could be other way?
6. Image Data - I'm pretty sure it would be easy to pass from Photoshop / AE into PB some basic information about the Image. Dimensions, Min Value, Max Value, Mean Value, etc...
7. Built In Functions - Colorspace Conversions (LAB / HSL).
8. Radio Buttons.
9. Vector Casting - Casting from Bool / Int / Float Vectors into Bool / Int / Float in one line. Sometimes it is needed to cast each item in the vector individually.
10. Defining DOD() - Sometimes Auxiliary images are needed. Let us create a DOD(). Define an array of RGBA in the size we want. This would make Resizing algorithms possible in PB. This could be even some kind of Auxiliary matrix for various calculations. Moreover, it's needed for many other uses (Building Matrices for other calculations such as Least Squares methods etc...).
11. Sample Nearest Mode B - In the current method ("Sample Nearest Mode A") if the Kernel access out of the DOD() pixel it gets (0, 0, 0, 0) as a result. In Mode B I would like it to actually return the value of the nearest pixel (Euclidean Distance), Basically "Padding" the image with its border. This would result in a much better Convolutions results.
That's it for now.
Would love to hear your opinions.