34 Replies Latest reply: Dec 21, 2012 11:48 AM by Lewiz RSS

    Question about CUDA on Nvidia 4000

    Lewiz Community Member

      Hi there everyone,

       

      In the process of getting my Premiere CS6 application to finally work with my Nvidia Quadro 4000 graphics card WITHOUT crashing immediately, as soon as the MPE is switched to "GPU accelerated" (in which I still haven't succeeded the last couple of months) I have reached the point where NO-ONE seems to be able to help me anylonger, which is why I am constantly testing new theories about this.

       

      It seems everytime I switch to involve CUDA power, an immediate time-out is caused and although I tweaked the Windows registry in many ways, with DWORDS and QWORDS with incredibly long delay times, the error persisted.

       

      Now I recently discovered that Nvidia runs on (at least) three processes: nvvsvc.exe, nvwmi64.exe and nvxdsync.exe, but I wondered why the first two of these appear TWICE when I select "Show processes from all users" in Windows task manager. See picture below. Could it be that some other app or even Windows itself is already "occupying" the CUDA cores, which causes the Premiere MPE to time out on it? And if not, why do nvvsvc.exe and nvwmi64.exe both run twice with different quantities of memory?

       

      Capture.JPG

      Anyone running Premiere CS6 with an Nvidia Quadro 4000 seeing the same processes? Very curious about this one, so thanks a lot for your reply in advance.

        • 1. Re: Question about CUDA on Nvidia 4000
          Andrew_S Community Member

          I'm running win 7 and a quadro 4000. This is what I have in task manager;

           

          4000-driver.jpg

           

          Although the file nvwmi64.exe does exist in my C: drive (in two places), as you can see its not running in Task Manager. I'm also running an older driver version. Maybe that's the difference?

          • 2. Re: Question about CUDA on Nvidia 4000
            Lewiz Community Member

            Dear Andrew,

            thanks a lot for taking the time to answer. The difference is that I choose for a custom install of my Nvidia stuff and I deliberately chose the Nvidia WMI driver and NOT the Win7 WMI driver, which is probably the driver that you have running instead. Now to complete things, I would kindly like to ask you one more question:

            Can you see IF you have a Windows WMI service/process running and if so, WHAT VERSION that is? I could easily uninstall the NVidia WMI driver and install a Windows WMI driver instead and WOWW.... maybe this could solve this problem that lasted an eternity, in my perception. I will simply "imitate" your config and see if that works....

            Any answer more than greatly appreciated!

            • 3. Re: Question about CUDA on Nvidia 4000
              Andrew_S Community Member

              In Windows task manager in the processes tab the only wmi I can see is     WmiPrvSE.exe      described as     WMI Provider Host     . No version number shown.

              On the services tab the only wmi is     wmiApSrv      WMI Performance Adapter.

              • 4. Re: Question about CUDA on Nvidia 4000
                Lewiz Community Member

                Hi Andrew,

                I rebuilt my configuration with exactly the same driver you are using, with WMI not coming from Nvidia but Windows, but alas... to no avail.  simply HAD to try, thought it might be the solution. Too bad, never mind, I guess I have to be patient until CS7 or the moment Nvidia will FINALLY decide to bring out a partner certified driver for CS6, which is not yet the case.

                Thanks a lot anyway!

                • 5. Re: Question about CUDA on Nvidia 4000
                  shooternz Community Member

                  Could this be a power supply issue?

                  • 6. Re: Question about CUDA on Nvidia 4000
                    Lewiz Community Member

                    Hi shooternz,

                    The fact that the Quadro's fan rotation speed is boosted at the moment of every crash for a few seconds (because it is being reset every time) would make me suspect it could be a power supply thing... but then, Premiere CS FIVE works flawlessly on the CUDA cores, which suggests that Premiere CS SIX would make a very specific and different appeal on the card's cores. Could that be possible? Would that require a bit of tweaking in the mobo BIOS or the Power Management of the Nvidia control panel?

                    • 7. Re: Question about CUDA on Nvidia 4000
                      Alex - DV411 Community Member

                      The fact that the Quadro's fan rotation speed is boosted at the moment of every crash for a few seconds (because it is being reset every time) would make me suspect it could be a power supply thing... but then, Premiere CS FIVE works flawlessly on the CUDA cores, which suggests that Premiere CS SIX would make a very specific and different appeal on the card's cores.

                      It's puzzling.  One of my clients had a similar crashing issue (HP Z800, Quadro 4000, Premiere Pro CS5) and it was cured by replacing the Quadro card.

                       

                      Any chance for you to borrow another Quadro 4000 somewhere, or even a different Quadro card (even a Quadro 2000 or FX 1800 will work)?  If it crashes too, it's a system issue; if it doesn't - it's a card issue.

                      • 8. Re: Question about CUDA on Nvidia 4000
                        tfi productions 44 Community Member

                        hello all,

                         

                         

                        i have a win7 pc tower with an i7 930, 24gb ram and 2 quadro 4000's with 4 monitors

                        i run a very stable cs5.03

                         

                        the quadro 4000 cards run extremely hot (from pny, at least)

                        so hot that i have manually configured the quadro fans to run at 85% constantly

                         

                        when exporting with PPRO or AME on high gpu intensive jobs the 1 card processing the cuda

                        would heat past 100 degrees C then  crash the system...even with the gpu fans turned up to 100 percent

                         

                        my solution was to buy a stanley  blower fan

                         

                        now both cards idle at 70 degrees C and export at 85 degrees C (with the gpu load rising as high as 80 percent usage)

                         

                        i have tweaked my OS a lot and have configured my 'power profile' to be always at 100 percent Performance

                        that way i know my hardware is always giving me maximum power...which is why i think the quadros are always running hot...

                         

                        it is the next phase i want to trouble shoot is modifying power profiles to find a suitable one

                        that will allow the cards to only ramp up to maximum when i am exporting...

                         

                        i removed the side casings from my tower and added the stanley fan...which is very powerful and quiet

                         

                        hopefully this helps...

                         

                        i love adobe products...

                        i love my quadro4000s

                         

                        j

                         

                         

                        you should get a gpu monitor and starting checking your gpu temps

                         

                        perhaps as soon as you open premeire the card goes into high performance mode

                        because of the gpu acceleration potential...the card heats up (and they heat quickly)

                        and your system crashes...

                        • 9. Re: Question about CUDA on Nvidia 4000
                          Harm Millaard Community Member

                          These are extreme temperatures. I have a GTX 680/4GB overclocked by 15% and the idle temperature is around 35C and does not rise to more than 58C under heavy load. I think you ought to investigate your cooling and airflow in the case.

                          • 10. Re: Question about CUDA on Nvidia 4000
                            Lewiz Community Member

                            Hi everyone, thank you so much for bringing in new possibilities to investigate. I took a little time today to have a monitoring app checking my GPU. It appears normally, the temperature varies a bit around 60-65 degrees Celsius. When Premiere is started, the temp goes up to around 74/75 degrees, where it remains stable. As soon as I power up the GPU acceleration from within Premiere, Premiere crashes and Windows reports the wellknown Display driver error:

                            WindowsError.JPG

                            Followed by the nVidia error:

                            ErrorNVidia.JPG

                            Premiere is still invisibly active and has to be terminated through the use of the Windows task manager.

                            GPU monitoring shows no temperature peaks of any kind, no heavy loads, there is nothing in the log files whatsoever.

                             

                            I dare to doubt it's a temperature or graphics card issue, because my workstation was built by a professional workstation engineering company, but there is something pretty untouchable about the CUDA part of this story. Premiere learns me that using CUDA or not is a matter of choice, the card runs fine without CUDA functioning. So if the CUDA capabilities of the card would be somehow defective, this would not be particularly easy to notice at all, would it? How can I find out, WITHOUT using Premiere for that purpose, that CUDA is working at all? Is there a way to check how the MPE makes its appeal on CUDA? I always presumed CUDA was working fine in the previous Premiere version, as I was able to switch the MPE on and off without any problem... but I never benchmark-tested the difference. Maybe CUDA never worked but this simply never shows because the only program making use of it, in my case, is Premiere ??

                            • 11. Re: Question about CUDA on Nvidia 4000
                              Harm Millaard Community Member

                              The temperatures worry me. It is way too high to start with when idle and they rise much to fast. On my system the idle temperature with a large or complex project loaded the temps remain around 35° C. In contrast your 74-75° C are extremely high, more than double my normal temps. When rendering extremely complex timelines with 7 tracks, that even on my system take around 1 minute to render with hardware MPE turned on, the temperature of the video card, despite the overclocking does not rise to more than 40° C.

                               

                              I don't care who built the system, you have a serious cooling problem.

                               

                              If you want to see if that really is the case, go to http://ppbm7.com/index.php/instructions and download the file. Unpack the ZIP file, open the project and with hardware MPE turned on, render the H.264 timeline and report what temps your video card shows during the render.

                              • 12. Re: Question about CUDA on Nvidia 4000
                                Andrew_S Community Member

                                Harm, what do you use to monitor the temperatures?

                                • 13. Re: Question about CUDA on Nvidia 4000
                                  Harm Millaard Community Member

                                  I use either GPU-Z or EVGA PrecisionX. In this case I used EVGA PrecisionX to monitor the temps while PR was rendering that devilish timeline. It is easier to read than GPU-Z.

                                   

                                  @Lewiz,

                                   

                                  Can you also report the idle temps of the CPU, motherboard and disks using HWMonitor and the same while rendering this timeline.

                                  • 14. Re: Question about CUDA on Nvidia 4000
                                    Lewiz Community Member

                                    Hi Harm,

                                    now that would be truely wonderful: being able to render any timeline with the MPE turned ON... that's the problem, you see, when I turn on the MPE Premiere explodes in less than two seconds, without touching anything. Simply turn on MPE, count to two and... Boom. I am sorry to say I cannot render this test with CS6, but I could try it with CS5 and see what it does. Thanks for the EVGA PrecisionX advice, I'll give that one a try. Maybe my temperature measuring wasn't accurate

                                    • 15. Re: Question about CUDA on Nvidia 4000
                                      Harm Millaard Community Member

                                      Lewiz,

                                       

                                      If you start with only software MPE and start rendering, you will get a good indication of what temps your video card will attain. You can cut off the test after a couple of minutes and report the other temps as well, otherwise you may be in for a very long wait.

                                       

                                      That info may be helpful in locating the problem.

                                      • 16. Re: Question about CUDA on Nvidia 4000
                                        Lewiz Community Member

                                        Hi Harm,

                                         

                                        as for the Premiere CS6 test file: this file is probably saved with MPE ON (it's a project related setting) so of course, directly upon loading, Premiere crashes straight away. I don't even have a chance of switching the MPE off. So I cannot open this file. I tried my best with a complex timeline I built myself, but the rendering does NOT make the temp go up more than 5 degrees. It starts hot, it stays hot, it doesn't overheat.

                                         

                                        As for EVGA PrecisionX (strange detail: they don't mention NVidia at all in their supported hardware list) I installed it and saw my first measuring results were accurate: the temp rises to around 70 degrees Celsius when starting Premiere. Thanks to PrecisionX I was able to force that temperature down by no less than 20 degrees to around 50 degrees Celsius, with Premiere started. I switched the MPE on and BOOM as always; Premiere crashes but the log file shows NO temperature excessions at the moment of the crash, neither did the GPU monitor.

                                         

                                        My conclusion would be that the temperature levels inside the PC could of course be improved a little, but they are absolutely acceptable - as seems also to be proven by the fact that Premiere CS FIVE has never had a problem whatsoever, MPE on or off. I don't think the temperature is the cause of the NVidia error 3 on Premiere CS SIX, but in due time I could enhance temperature control a little just for hardware's sake.

                                         

                                        Anyway, it seems to me like looking in the wrong direction... what I want to share is something I should have shown in the first place; what do you make of this:

                                        GPU.JPG

                                        Seems to me like 475 Mhz isn't like top-notch at all, but...? Maybe I am missing something? On the other hand, Quadro 4000's should be pretty much all the same, shouldn't they...

                                        • 17. Re: Question about CUDA on Nvidia 4000
                                          Harm Millaard Community Member

                                          You could try a new software MPE project and import the H.264 sequence into it. Or force MPE to off by modifying cuda_supported_cards.txt to read not Quadro 4000 but changing that line to read Quadro 400 or deleting the line with Quadro 4000 altogether.

                                          • 18. Re: Question about CUDA on Nvidia 4000
                                            Lewiz Community Member

                                            I'll give it a go

                                            • 19. Re: Question about CUDA on Nvidia 4000
                                              Lewiz Community Member

                                              OK, a little news here.

                                              I decided to use EVGA PrecisionX to overclock the NVidia Quadro 4000, just for testing purposes. When no single app is running, the GPU clock is at 50 MHz. As soon as I doubleclick Premiere.exe the GPU frequency shoots up to whatever set maximum; in my case this was 475 MHz on GPU and 1404 MHz on the memory. This made Premiere crash as usual. Then I pulled the overclock slider to the max it could give: 760 MHz GPU, 1684 MHz memory.

                                               

                                              Guess what happened? No, the temperature was kept completely under control by PrecisionX. No noticeable change. The higher the temp, the faster the fan - no problem.

                                               

                                              But... PREMIERE DID NOT CRASH for the first time since ages. I couldn't get any video on screen or external monitor, but the sound worked ok and the timeline was playable and scrubbable and seemed pretty stable. JUST NO VIDEO,I know, what is an editing app without video, but still, this puzzles me even more.

                                               

                                              It suggests that first of all, the standard clock speed of 475 MHz is way too slow for the "new Premiere", version CS6, whereas it always proved sufficient for CS5. I can now understand why the error was comparable to a Windows time-out. The card's simply not fast enough. When I overclock it, the timeout crash is gone, but it is obviously still way too slow for CUDA operation, so there is no image output.

                                              THIS IS NOT a temperature issue, it has something to do with tweaking the NVidia 4000. My graphics clock can not go past 760 MHz, memory limited at 1684 MHz - with MPE switched OFF the maximum speed is already claimed... when MPE is ON the NVidia 4000 simply stops processing video because it's too much. And then I mean just one layer of SD video with no effects...

                                               

                                              Now NVidia is pretty specialized in enhancing 3D apps and games - but where it comes to programming the CUDA cores for video editing.... how good is that? In my NVidia Control Panel I have a switch called "Enable VIdeo Editing Mode" under the "Desktop" menu, but I can click that until I drop and it shows no tick at the tickbox neither does it change the least in the card.

                                               

                                              I will do what I can to get my hands on another graphics card.

                                              • 20. Re: Question about CUDA on Nvidia 4000
                                                Harm Millaard Community Member

                                                Lewiz,

                                                 

                                                I never had a Quadro card, but generally these are made to rather tight specs. If overclocking does help to avoid the crashes, I suggest you try out very moderate overclocking, like a GPU of 525 MHz and a memory at less than 1100 MHZ to start with. You can always fine-tune later, but first get the video in order.

                                                 

                                                BTW, did you adjust the fan curve to reduce the temps to get it back from 75 to around 50° C?

                                                • 21. Re: Question about CUDA on Nvidia 4000
                                                  wrinklypeter2

                                                  Hi

                                                   

                                                  I've just 'upgraded' from a GTX 285 to the Quadro 4000 - primarily because I was getting display crashes when using the Hardware acceleration in Premiere CS6. Now, after the upgrade, that has completely disappeared and the graphics are working fine.

                                                   

                                                  I have checked the processes running and each of the 3 processes are only shown once. I'm using the latest drivers - installed yesterday - and everything works fine.

                                                   

                                                  Can't offer an explanation for the crashes you're getting - sorry!

                                                   

                                                  Very disappointed with the quadro 4000 performance on AVCHD editing - not obviously better / faster than the 285 GTX.The hardware CUDA performance is no better than when I use software rendering in premiere CS6 - at least not when editing HD264 (which I am at the moment). Haven't checked against SD editing though.

                                                   

                                                  I'm running an I7 3.2gig with ssd (main) hard drive, raided video drive and the win7 benchmark shows a performance score of 7.4 from 7.9 - with the Quadro 4000 producing the lowest score at 7.4.

                                                   

                                                  Any idea how I might improve speed of rendering etc

                                                  • 22. Re: Question about CUDA on Nvidia 4000
                                                    Andrew_S Community Member

                                                    Below are two screen shots from GPU-Z of my quadro 4000 card if this helps in the discussion of temps and frequencies.

                                                     

                                                    CPU-switch -on.jpg

                                                     

                                                    CPU-switch -on 2.jpg

                                                    • 23. Re: Question about CUDA on Nvidia 4000
                                                      wrinklypeter2 Community Member

                                                      Hi Andrew,

                                                      Thanks for the TechpowerUp tables.

                                                      I installed TechpowerUp and got very similar results to your shown - except that my GPU temperature was a bit higher AND (this is the query!) I find that when I start TechpowerUp it reports CUDA in use. However, as soon as I load premiere CS6 and then review re-start TechpowerUP it shows that CUDA is not in use.

                                                      The premiere project is using the Mercury Playnack hardware acceleration, and i'm editing in h264 mode.

                                                      How can I correct this? Any ideas would be appreciated.

                                                      • 24. Re: Question about CUDA on Nvidia 4000
                                                        Harm Millaard Community Member

                                                        I find that when I start TechpowerUp it reports CUDA in use.

                                                         

                                                        Not in use, but supported. It is up to the application to effectively USE it or not.

                                                        • 25. Re: Question about CUDA on Nvidia 4000
                                                          wrinklypeter2 Community Member

                                                          Ok. Point taken.

                                                          However, if CUDA is being reported as not supported in Premiere CS6 then surely this is incorrect and I need to take some action to correct this?

                                                          • 26. Re: Question about CUDA on Nvidia 4000
                                                            John T Smith CommunityMVP

                                                            I'm not sure if that card is automatically supported or not, but have you checked the file mentioned in the "hack" message?

                                                             

                                                            http://forums.adobe.com/thread/629557

                                                            • 27. Re: Question about CUDA on Nvidia 4000
                                                              Lewiz Community Member

                                                              Hi Harm,

                                                               

                                                              Yes, I tweaked the fan curve. I tried 525 MHz and less than 1100 memory, but to no avail.

                                                              Only on MAX overclocking the NVidia won't crash... but then it still doesn't give video output at all.

                                                               

                                                              Found something interesting though using TechPowerUp (see Andrew_S's reply below): hovering over the CUDA checkbox a balloon pops up saying:

                                                               

                                                              Supported compute capability: 2.0

                                                               

                                                              Does this mean CUDA version 2.0 ? That would be pretty outdated, because we're currently at 5.0 right? Hmmm still puzzled beaucoup plus.

                                                              • 28. Re: Question about CUDA on Nvidia 4000
                                                                Lewiz Community Member

                                                                Hi Andrew,

                                                                 

                                                                I am not getting the CUDA checkbox unchecked when running Premiere with a MAX overclocked Quadro 4000, my temperature is significantly higher, but I can tweak that and still have the crashes, so temp isn't the problem.

                                                                I do have another BIOS version  but I have been warned not to change it - and even if I would want to, this version you have and any kind of BIOS update is unretrievable.

                                                                 

                                                                Still puzzled...

                                                                • 29. Re: Question about CUDA on Nvidia 4000
                                                                  Lewiz Community Member

                                                                  Hi John,

                                                                  yes, I "hacked" it, gets the MPE CUDA acc. mode out of the grey and clickable, but crashes still.

                                                                  • 30. Re: Question about CUDA on Nvidia 4000
                                                                    Andrew_S Community Member

                                                                    Hi Lewiz, "hovering over the CUDA checkbox a balloon pops up saying:   Supported compute capability: 2.0"

                                                                     

                                                                    I get the same.

                                                                     

                                                                    As for the screen grab images in post 22, that's not the full story! Those grabs were taken about 10 mins after the PC was switched on. When I play a video in VLC or run Premiere, the temp rises. Its been up to 72 C so far. The system here is a HP Z400, so I assume that HP do know what they are doing with regard to cooling.

                                                                    There's no over clocking in use here. I just installed the Nvidia drivers and that was that.

                                                                    • 31. Re: Question about CUDA on Nvidia 4000
                                                                      wrinklypeter2 Community Member

                                                                      Sorry to appear stupid, but when I run GPUSniffer I can see the following:

                                                                       

                                                                      GPU Computation Info

                                                                      Found 1 devices supporting GPU computation

                                                                      CUDA Device 0 -

                                                                      Name Quadre 4000

                                                                      Capability: 2

                                                                      Driver: 5

                                                                      Total Video Memory: 2047mb

                                                                       

                                                                      Can I then assume that Premiere CS6 IS using (or has at least recognised!) the Quadro card?

                                                                      - even though GPZ does not recognise the CUDA as being supported when Premiere is running?

                                                                      P.S. I also have the 'Supported compute capability 2.0' baloon.

                                                                       

                                                                      Thanks for any help!

                                                                      • 32. Re: Question about CUDA on Nvidia 4000
                                                                        Lewiz Community Member

                                                                        Hi Andrew, that's what I expected. 40 degrees sounded like a disemployed GPU. Remarkable how Harm does this; even when my GPU is almost completely idle it will stick around 65 degrees. Although I must add I saw the temp going up to 85 a couple of times.

                                                                         

                                                                        And indeed, these relatively high temps are accomplished without overclocking. Good to hear you have this same 2.0 capability, I can scrap that idea.

                                                                         

                                                                        There is something with flashing the BIOS however, it seems there are no resources whatsoever on internet enabling the download of any kind of Quadro 4000 BIOS or giving in-depth info. All I seem to know about it now is that each BIOS has its own hardware demand in terms of maximum and minimum limits of overclocking, voltage etc. which is why a wrong BIOS could damage the card, which on its turn makes it dangerous to install without sufficient knowhow, which is probably why BIOSes aren't sread all over the internet.

                                                                         

                                                                        But I did notice you had a different version than I have.

                                                                        • 33. Re: Question about CUDA on Nvidia 4000
                                                                          Lewiz Community Member

                                                                          @wrinklypeter: it means Windows sees the card and its capabilities, nothing about actual usage. GPUSniffer will even give this report before you have ever installed Premiere

                                                                          • 34. Re: Question about CUDA on Nvidia 4000
                                                                            Lewiz Community Member

                                                                            Dear, dear people: The endless quest has come to an end.

                                                                             

                                                                            It seems now that some of you have been pointing in the right direction, although the very details of the cause of this problem are still a little mysterious to me.

                                                                             

                                                                            Today I received a parcel from the company that built my system. The contents: a brand new Quadro 4000 card. I swapped it with my old Quadro 4000 and everything works like a charm now.

                                                                             

                                                                            I am still puzzled how it can be possible that, at any given moment, I could start up Premiere CS FIVE and experience no CUDA problems at all, while working with Premiere CS SIX appeared impossible - seemingly due to a proven hardware problem now. But then, the CUDA was still powerful enough for older versions of Premiere? I am still amazed that a graph card can be partially broken... and the defect only becomes visible with newer software... weird.

                                                                             

                                                                            Anyway, I wish to thank all of you here for helping me with the best of your knowledge and ideas. I am sorry to have wasted so much of your time with something that was as simple as replacing the hardware.... shame on me.

                                                                             

                                                                            I'd like to take this opportunity to wish you all an incredibly merry Christmas and a very inspiring, creative New Year ! So long!

                                                                             

                                                                            Regards,

                                                                             

                                                                            Tom Verweij