16 Replies Latest reply on Jan 14, 2014 7:49 PM by ikatz002

    What impacts performance the most for big files

    ikatz002 Level 1

      Hi,

       

      I primarily use PS to edit very big images, native resolution about 250 megapixels (about 1GB raw tiff file at 16bit).

       

      I apply many adjustment layers and filters to these images. I am wondering what part of the system is most affected for applyting Filter Gallery effects such as Paint Daubs, Glass, etc.

       

      I currently have a pretty beefy workstation with top of the line dual Xeons, 32gb or RAM and a AMD Firepro V7900 (with 2GB Ram)

       

      I am finding applying filter effects takes a very long time with these images, it is clearly not the processor since it is at about 6% usage when applying the filter. It is also not the RAM since it usually has about 5 more GB to go before reaching the top. I have trouble believing its the video card becasue im not asking it to do anything real time, it seems more of a processing job (applying the gallery filter effect that is).

       

      So my quesiton is, what can I do to better the performance of applying these effects? where exactly is the bottleneck here? Any suggestions are very welcome and appreciated.

       

      p.s. yes I have my performance settings optimized (big and flat, advanced rendering mode, plenty of RAM for PS to use, etc.)

        • 1. Re: What impacts performance the most for big files
          Trevor.Dennis Adobe Community Professional (Moderator)

          I think you'll find the Scratch disk space is being used a fair bit with such files.  I have 32Gb RAM and see big scratch file with much smaller image docs.  So a fast disk reserved just for scratch space and maybe cache for some of your apps.

           

          What settings are you using in Preferences > Performance ?  I'd go for 6 cache levels and 1024k tile size with such big files. 

           

          Also check out the GPU FAQ, (I am about to go out so you'll need to Google)

          • 2. Re: What impacts performance the most for big files
            JJMack Most Valuable Participant

            250,000,000 Pixels where did these pixel come from.  Where they made by stitching many images together to form a detailed high resolution image or are your images just interpolated up in size to increase the number of pixels.  If the latter your shooting yourself in the foot.

            • 3. Re: What impacts performance the most for big files
              ikatz002 Level 1

              Hey Trevor,

               

              Thanks for your reply. I am using a superfast SSD set aside for the scratch disk. I also have 6 cache levels at 1024k

               

              I checked the GPU faq and didint find anything useful, they just tell you to put the rendering mode in advanced if you a powerful video card, which ive always had at.

               

              That being said I dont think the slow performance im having is due to writing on disk. It may be due to the GPU, but im thinking its probably due to poor coding in photoshop not taking advantage of all the processing power available to it (multithreading, etc.) But there is no real way for me to know unless someone from adobe tells me - what is most being used as far as computing when applying filters to images or changing the blending mode of those filters?

              • 4. Re: What impacts performance the most for big files
                ikatz002 Level 1

                They were computer generated

                • 5. Re: What impacts performance the most for big files
                  Level 7

                  Filter Gallery: fast disk, fast CPU (most of it is not threaded).

                  • 6. Re: What impacts performance the most for big files
                    ikatz002 Level 1

                    Hey Chris,

                     

                    Thanks for your reply. I have the scratchdisk on an Corsair Force GT SATA III which claims write of 455mbs and read of 540mbs. I have dual xeons and as I mentioned, when it is applying the filter I can see the CPU ramp up to about 16% usage on all threads evenly. (not like it is only using one core to the max and leaving the others unused, which happens on programs which do not support multi-threading)

                     

                    In any case, using the resource monitor I watched as I applied a filter and there was no disk activity, so I doubt it has anything to do with the disk..

                     

                    Specifically Photoshop is slow on two occasions, when it is preiewing the effect in the gallery effects window (ie. rendering tile by tile), and when it is applying said effect (and there is a progress back saying 'applying X effect')

                     

                    On both those occasions there is no disk activity, the CPU is barely being used and there is still plenty of RAM left.. so my question is - what is photoshop using and what can I change in my computer hardware wise in order to improve it.

                     

                    Thanks again!

                    • 7. Re: What impacts performance the most for big files
                      Level 7

                      CPU is "barely used" because only one core is being used, but that core is very busy.

                      To make it go faster, you'd need a faster CPU.

                      • 8. Re: What impacts performance the most for big files
                        Trevor.Dennis Adobe Community Professional (Moderator)

                        Chris, what happens in terms of cores and threads used, with dual CPU systems?  I suspect you are going to say just one thread of one CPU, and the other not used at all.

                         

                        ikatz,  If your images are computer generated, can you not up res them near the end of the process?  Depends on the image of course.  Can you upload a taster?

                        • 9. Re: What impacts performance the most for big files
                          ikatz002 Level 1

                          Hey Chris,

                           

                          I beg to differ. As mentioned above, all threads are being used equally when applying the filter. I dont know to what extent photoshop can take advantage of multiple threads or cores, but I do know the behavior of a program that can only use one core and PS is not it, basically I can see one core being used to 100% while the others remain idle. This is not what happens when I apply filters or use PS in general, and I can upload screenshots to prove it.

                           

                          I go back to my original question - what is it that is most being used when applying a gallery filter in terms of computation, is it the CPU, GPU or something else? (and if it is the CPU, why doesnt it cycle to full usage?)

                           

                          Trevor, I output the final render and work on that, there is no way to go back and forth. I need the high resolution because these are printed in large format. If you want to take a look you can visit my page at www.isaac-katz.com

                          • 10. Re: What impacts performance the most for big files
                            JJMack Most Valuable Participant

                            As Chris has pointed out some processes do not lend themselves to be multi threaded in those cases only one core on your dual xeons will be be used to do the processing. So the processor speed is what is most important. 

                             

                            I'll ask again where are the 250,000,000 pixels your images seem to have, come from. Are you stitching 20 or more images together to create a high detailed high resolution image or are they just images that have been interpolated up in number of pixels.  If the later your interpolating your image up in size too early in your processing.  You should work with your images original pixels till your ready to output you final image file.  Then Interpolate for you output size.  Processing 250,000,000 lower quality pixels slows down processing and gains you nothing.

                            • 11. Re: What impacts performance the most for big files
                              ikatz002 Level 1

                              Hi JJ,

                               

                              The 250mp are native resolution, i dont interpolate ever. They are computer generated, which means if I want to (and have the time) I could render a million megapixel image (and yes, it would have detail up to that resolution).

                               

                              Please read the answers I have already submitted before posting the same question. I can easily see on my task manager the functioning of my cores, and I can easily tell how photoshop is using them. It is not using only one core, it is using them all evenly, but at a low % of their capacity. Which makes me think the bottleneck is somewhere else. (and no, there is no way task manager averages the use of one core and shows me the ditributed processing over all of them, there are single thread programs that use only one core, and I can plainly see one core being used at 100% while the others remain virtually at zero, this is not the case in PS).

                              • 12. Re: What impacts performance the most for big files
                                Noel Carboni Level 8

                                Windows can and will reassign different cores to run a particular thread if the thread makes system calls.  What you're seeing is exactly consistent with a single-threaded application running and making such calls.

                                 

                                Photoshop contains legacy code going literally back to the late 1980s.  Not all of it has been modernized to use modern multi-core machines, nor optimized to make the best use of modern caching systems.

                                 

                                Regarding opening and saving files slowly, look into changing your settings for compression.  Compression and decompression are activities that do not lend themselves to multi-threading either.

                                 

                                You should know that Chris is one of the high-level engineers who actually codes Photoshop.

                                 

                                -Noel

                                • 13. Re: What impacts performance the most for big files
                                  JJMack Most Valuable Participant

                                  Perform a small test for me. Open one of your 250MP images. If layered create a composite layer  Stamp visible layer into a composite layer. Then use filter Smart Sharpen on that layer while you have a CPU monitor running so you can see a filter that uses many threads see if you use more the 6%It should take some time 250MP 16Bit a lot of bytes to process. My CPU hit something like 40% Photoshop Scratch file reached to 23GB and ram Usage hit 18GB.

                                   

                                  You say the pixels are computer generated.  And you can generate as many as you would like.  Why do you like to generate a 250MP image how are you using this document.  Why do you feel you need 250MP?  To me computer generated read a lot like interpolating something.

                                  • 14. Re: What impacts performance the most for big files
                                    ikatz002 Level 1

                                    Hi Noel,

                                     

                                    Thanks for your reply. So are you saying this is a pure CPU issue and has nothing to do with my GPU, so upgrading video cards will not affect at all that aspect of performance?

                                     

                                    I did not mean to say that I know more than Chris by any stretch.. its just strange because I have seen aplications that are single threaded use only one core and its clearly visible in the task manager when they do so. When PS works the work is spread evenly along all the cores, which is different. But I understand that it may not be coded as to take full advantage of all the cores.. Just want to know if there is something beyond changing CPUs which can affect that part of performance.

                                     

                                    Thanks again and thank you Chris for taking the time to answer

                                     

                                    p.s. I do not have issues opening and saving files.. dont know where that came from.

                                    • 15. Re: What impacts performance the most for big files
                                      Noel Carboni Level 8

                                      I just threw in the compression/decompression comment because of the subject of the thread - you mentioned dealing with "big files".  As a computer geek the term "big file" to me says perhaps a different thing than it does to others.

                                       

                                      What it appears you're seeing is performance limited to one or at best a few threads crunching through certain things, which will be most affected by the speed of one core of your processor.  Chances are good that won't change with a video card update.

                                       

                                      But you said, "I am finding applying filter effects takes a very long time with these images"...  What filter effects, specifically? Are you talking about effects from the Filter Gallery?  Or other things?  You may not realize that not every filter/plugin/feature of Photoshop runs through the same code, but it would be closer to accurate to say each has its own implementation.

                                       

                                      It's possible Adobe is working to improve performance on the specific things you do and need (as Chris says, they refactor things from time to time, and especially lately they're moving a lot of processing to the GPU in recent releases).  It's possible that they're not refactoring the specific filter you're working with but might consider doing so if they knew.  They do listen.  Being specific will help that.

                                       

                                      -Noel

                                      • 16. Re: What impacts performance the most for big files
                                        ikatz002 Level 1

                                        Hey Noel,

                                         

                                        I agree. With most thing PS runs like a breeze (as should be expected by my set up), the only areas where I notice it takes a long time to respond, as I mentioned earlier in the thread, are: applying gallery filter effects (both the tile rendering in the preview window as well as when actually 'applying') and when changing blend modes on the effects (usually stacked effects on a single smart object layer).

                                         

                                        Beyond that, it happens on every effect ive applied from gallery filters. I havent tried all, but ive tried most of them.

                                         

                                        Just as a note, exactly when I have 4-5 effects stacked on a smart object layer, if i change them around or change their individual blend modes, the message comes up 'rendering smart filter' and its by far when the stuff takes a loooong time to do. ie. when i notice the biggest performance hit in PS. (yes this smart object layer is around 250mp in size, so I imagine it would be though

                                         

                                        Hope that helps.