When I try to use Pixel Bender OilPaint on larger images( PSD or JPG), I get this advisory:
"Warning: Storage requirement for complex filters may overload GPU when processing large images. Please switch to CPU processing in issues are enountered." When the box for GPU is checked, I get a black screen. When I uncheck it, there are just garbled pixels. I haven't been able to find info on making the switch from GPU to CPU. Can it be done? Or do I just have to work with smaller images? Thanks for any info.
Unchecking the "Process on GPU" checkbox will process the image on the CPU. You should not get "garbled pixels" but maybe another error is occurring. You might try leaving the "Process on GPU" checkbox disabled, switching to another filter, and then switching back to OilPaint. What are the dimensions in pixels of the images you are trying to process? What OS are you using? How much memory is installed in your machine? What graphics card are you using and how much video memory is on the card?
Thanks for the info on going to CPU by unchecking the GPU box. I tried going to the filters, and then to OilPaint and while the other filters work (in GPU -- going to CPU makes things worse), with the same black screen appearing. All is fine at smaller image sizes. The image I am working with is 16x20 inches, 300 resolution. I am working in Mac OS X v. 10.6.8, with 8 GB of memory. Graphics card is NVIDIA GeForce GT 120 with 512 MB VRAM. I'm guessing now that more memory in one of these places is the answer ... yes?
I would expect this to process correctly on the CPU although I would not expect it to be particularly fast. Are you running Photoshop in 64-bit mode? If not, you should give that a try. We do final render on the GPU regardless of whether the filter processing was done on the GPU so it's possible something is failing along the final render path. For the sake of experimentation, you might try applying the effect to the image (click OK to exit the plugin) using the CPU even though the preview is black. I know that won't help you much but it will help us diagnose the problem. I don't believe adding more RAM will solve the problem at this point unless by adding more RAM you mean buying a newer video card with more video memory. That's not something I'd recommend unless you're absolutely flush with cash and looking to blow it on hardware as you have a decent graphics card and your upgrade options are severly limited on the mac. You might try driving the Pixel Bender plugin (and the OilPaint filter explicitly) from outside the plugin via Extendscript or Photoshop actions. You might also try segmenting your image into smaller pieces, filtering those pieces, and then recompositing the pieces into a final image. The problem with that is that OilPaint's edge effects will likely produce noticeable seams along the edges that make recompositing the individual pieces difficult. You should also try entering the plugin in CPU mode--apply any filter to the image, undo the result, and then relaunch the plugin--and then switching to OilPaint so that you are never in GPU-processing mode. It's unlikely this will workaround your problem but it's worth a shot. You might also consider reporting the issue to Apple's tech support as the GPU side of the problem feels like a driver bug. I'll let you know if I come up with other suggestions and please let us know if you find a way to workaround the problem. Best of luck.
I have put Photoshop into a 64-bit mode, and this was a promising step. The image would actually appear in the Pixel Bender window for OilPaint, but the controls are all wonky. You move a slider, and it doesn't slide, at least not until several moments of seeting the time-piece and the time swirl, then it may actually move, only to snap back to its original position. I just tried it with a different image sized the same way 16x20, 300 resolution with same results. I got to 64-bit by selecting the CS5 application file from the CS5 folder and unchecked "Open in 32 bit mode," which the tutorial said would then make it 64 bit. I'm not sure how this differs from going to Image>Mode>bit selection, which only goes up to 32 in this manner. I tried for a while to use the CPU mode, but kept getting advisories that this is taking too long and would you like to cance?. For a number of tries between messages I soldiered on, but those messages became so frequent that gave up on CPU. Bottom line: the picture is big (16x20 inches, 300 resolution), and somewhat smaller images work. Have I gone about a far as I can go? Thanks for earlier response.
Yes, the mode that Photoshop is launched in, 32- or 64-bit, is separate from the mode of the image, 8-, 16-, or 32-bit and should not be confused.
I think your best option at this point is probably to go the scripting route. You can drive Photoshop and the OilPaint plugin from ExtendScript. The best way to get started with scripting Photoshop is to use the "ScriptingListener" plugin included in the Scripting directory which is located at the root of the Photoshop installation directory. Search for "ScriptListener" in the "Photoshop CS5 Scripting Guide" for information about how to use this plugin. You can playback scripts, once you've recorded them, from ExtendScript and customize them to do what you want. You may even be able to process on the GPU through this path. You'll lose the interactivity that comes from scrubbing parameters in the Pixel Bender plugin window but it sounds like that's not working for you to begin with so you're no worse off than when you started.
You might also try applying OilPaint, in CPU mode and with the parameters set to their desired values, to a smaller version of your image and then reapplying the filter (Command + F) to the actual image thus avoiding the plugin dialog.
Finally, you might try leaving the "Processing Timeout" dialog onscreen for five or ten minutes before clicking through it to see if processing has completed as I believe the dialog does not block processing. I'm not 100% certain whether we continue to process in the background so give it a try.
I'll let you know if I think of anything else. Keep us posted on your progress.
Thanks for working with me on this. I had success with processing a smaller version of the image and then using the Command + F method. The Processing Timeout idea hasn't worked yet; it's still toiling away after about 45 minutes. I did install the ScriptListener into the Automate file and understand the concept of it, where in it records actions. But I didn't understand how to apply it to the Pixel Bender function. It seems at best I could record a sequence that would take me to the same stumbling block I can get to directly. I also tried an earlier suggesting, dividing the image into two parts, processing them, and them merging them back together. That worked without any seam showing. So at least I now have two options to use this tool. Thanks.
Glad to hear you've got a few workarounds
With regard to the "timeout," are you saying you clicked okay to dismiss the dialog after forty-five minutes and then another dialog was presented or are you saying that the original dialog was still onscreen after forty-five minutes? How long does it take to process the fullsize image using the "Command + F" workaround and are you processing on the CPU or GPU in that case?
I suggested the scripting option as a more powerful version of the "Command + F" workaround. You would be able to experiment with OilPaint and its parameters without having to invoke the plugin dialog. I think you can just stick with the "Command + F" workaround if it is working well for you; no need to spend any time on the scripting workaround at this point.