27 Replies Latest reply on Oct 31, 2013 5:21 PM by Alex - DV411

    Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?

    CinematicFusion

      I tried the K20x in my workstation and it did great with the exception that it overheated and I had to return it. I don't think that passive cooled card works well with the HP z820 workstation. Maybe it was just a bad card but it has scared me away from that card. Shame though, it was fast.

       

      Which dirction would you guys go?

        • 1. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
          Bill Gehrke Adobe Community Professional & MVP

          I guess you are running CC version.  Do you need the 10-bit output from another card, it does not appear that you do ?  If you do then maybe a K5000.  If you do not then you should probably try a GTX 780 and save a lot of money and out perform the k5000 on Premiere by far.  Verify that both cards can use the same driver.

          • 2. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
            cc_merchant Level 4

            See Tweakers Page - Balanced Systems and scroll down to the Quadro price/performance info.

             

            Then go to Tweakers Page - Exporting Style and see what you can expect from a video card. I have yet to see a situation where a Tesla card does anything more than empty your wallet. I have never seen a performance improvement.

            • 3. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
              Alex - DV411 Level 2

              I'd:

              1. Get another K5000 - if you want a short answer, but first I'd see if it'd do much vs. just one.
              2. check to see if the Z820 has both cardbay (low front area) fans installed. Those are great for passively cooled cards (and hard drives). Single CPU configurations usually have one, dual - two, and I always install both for clients.
              3. describe your workflow a bit (apps used, file formats, amount of rendering / encoding involved) - that would help determine the best GPU choices for you. Agree with both previous posters that there're likely good choices for less money, but still...
              • 4. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                CinematicFusion Level 1

                I already have a k5000 and I hear mixing gtx cards don't work. So, I can either add another used k5000 or sell the k5000 I have for another gtx config.

                 

                My workflow.

                I use all adobe products from premiere, ae, audition, photoshop.

                Eyeon fusion for complex comps.

                Realflow for fluids...not all the time, just when job requires.

                Mostly in max and use thinking particles and pflow.

                Vray for renders.

                 

                Sent from my iPad

                • 5. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                  ECBowen Most Valuable Participant

                  You can mix a Quadro and Geforce card. You just have to install the Geforce driver a certain way.

                   

                  Eric

                  ADK

                  • 6. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                    Alex - DV411 Level 2

                    I use all adobe products from premiere, ae, audition, photoshop.

                    Eyeon fusion for complex comps.

                    Realflow for fluids...not all the time, just when job requires.

                    Mostly in max and use thinking particles and pflow.

                    Vray for renders.

                    Sounds like a good Quadro card will be a better choice given it's higher (I think) FP math performance and ECC memory. Does 10-bit deep color support matter?

                     

                    I'd put Adobe apps aside and check and see what's best for other apps you mentioned. Can these apps benefit from dual GPUs? Would a single K6000 be better than dual K5000? Would adding that Tesla card (if it works) be better than another K5000?

                     

                    I'd also not mix Quadro and GeForce cards in this system for the risk of breaking something even if it's possible.

                    • 7. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                      CinematicFusion Level 1

                      I'll start doing research on the other apps.

                      Here is another thought....

                      Would upgrading my xeon processor be the best improvement. 

                      I have a single xeon e5-2660 with 64gb ram.   I'm thinking of upgrading to the e5-2687w v2. 

                      Would that be the better option to upgrade first and then maybe upgrade the card.

                      • 8. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                        cc_merchant Level 4

                        The K5000 is more than enough for a single CPU. The benefit of a second card is marginal at most, given the low clock speed and the single QPI link. Adding another CPU will help much more, so your idea is sound. The question is, adding another E5-2660 is far less costly than getting 2 E5-2687W's. The extra clock speed helps all the time, but is it worth the price? That is for you to decide.

                        • 9. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                          ECBowen Most Valuable Participant

                          Fusion, Realflow, and Vray have GPU acceleration either Open CL or Ray Tracer. Which means card specs are the key and not other features of the Quadro. Realflow lists Multi GPU support but not the others. Installing and using the Geforce cards with a Quadro is not that hard. Just install the Quadro card normally with the Nvidia Quadro installer. Last someone checked the Quadro driver was also including a Geforce driver now. If that is not the case then download the Geforce driver. Start the installer so the Geforce folder extracts to the Nvidia folder. Then cancel the install on the next frame. Go to device manager and manually update the display driver for the Geforce card. Point the directory to search to the Nvidia Geforce directory. That will install the Geforce driver without overwriting Quadro installs. They run fine together. Octane people have been doing it for years and I have done it here.

                           

                          The E5-2687 V2 will definitely be far better performance than the E5 2660 because of the clock speed difference with GPU acceleration. 2.0GHz on the 2660 is really to low for any of this type of work. I would definitely suggest that change first. I would also suggest looking at the 2697 V2 if you have the budget. The 12 Core versus 8 would be more than worth it for this work.

                           

                          Eric

                          ADK

                          • 10. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                            Alex - DV411 Level 2

                            I have a single xeon e5-2660 with 64gb ram.   I'm thinking of upgrading to the e5-2687w v2.

                            Are you thinking HP or generic (Intel) parts? Intel parts might void HP warranty and you'll still need HP heat sinks. You will also need the opional 1125W PSU unless you already have it. Other than that, it's a sweet upgrade. (2687w v1 used to require liquid cooling, apparently v2 - doesn't.)

                             

                            Yes, I'd say it's a better upgrade than a 2nd K5000 unless you're running multi-GPU-accelerated apps all day long, and your GPU load is higher than the CPU one - which is rather unlikely.

                            • 11. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                              Alex - DV411 Level 2

                              Fusion, Realflow, and Vray have GPU acceleration either Open CL or Ray Tracer. Which means card specs are the key and not other features of the Quadro.

                              Differences in floating point precision, rounding errors, ECC memory make no difference in wireframe rendering?

                               

                              I don't dispute that $ for $, GeForce cards blow Quadros out of the water in most performance benchmarks. I only question that performance is all that matters.

                              • 12. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                CinematicFusion Level 1

                                Good points, I need to ask hp about that. See if I can upgrade through hp...if not, the e5-2660 dual will be my only option with a warranty. I have the 1125 psu.

                                 

                                Sent from my iPad

                                • 13. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                  CinematicFusion Level 1

                                  Liked the idea of the 12 core but was not sure how an uneven amount of ram per core would work?  Hyper thread at 24 cores, I have 64 gb of ram, so ram per core is 2.6.   I've always heard you needed a solid number of ram per core. 

                                  Maybe I'm wrong.

                                   

                                  Sent from my iPad

                                  • 14. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                    ECBowen Most Valuable Participant

                                    The Geforce cards supposedly have greater tolerance for errors than the Quadro cards so yes that could come into play. The ECC ram has been discussed in every fashion since the Quadro cards started shipping with it. ECC ram is key when you have ram that has to refresh 24/7 for long periods of time. In those situations, an error can develop during a refresh cycle. However in testing that time has exceeded 6 months or more. GPU's rarely use ram refresh for that long period of time. Most who use Quadro's turn the ECC off unless there will be an extremely long data calculation process that will take days. Outside of that the ECC has little chance of having an effect on the render process other than slowing the ram cycles down.

                                     

                                    Remember though we are talking about using both a Quadro and a Geforce card together. In that scenario the editor can switch the acceleration to the Quadro card if FP calculations have to be refined. Also on the applications that use more than 1 GPU the Quadro will already be in play anyway. The Geforce card is simply adding a major performance boost to the Quadro versus spending a ridiculous amount of money on the Tesla cards which seem to be very problematic with many media based applications.


                                    Eric

                                    ADK

                                    • 15. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                      ECBowen Most Valuable Participant

                                      Remember you want to leave ram available for player caching and other applications. This would mean you allocate 2 GB per thread with the 12 core and leave the rest for the application cache or other applications. An example of this is the Quicktime server. Since QuickTime is 32bit most applications have to integrate with QuickTime via network process. This means ram allocation for QuickTime is separate from the main player and application.

                                       

                                      Eric

                                      ADK

                                      • 16. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                        CinematicFusion Level 1

                                        Ok, that makes sense.

                                         

                                        Sent from my iPad

                                        • 17. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                          Alex - DV411 Level 2

                                          The Geforce cards supposedly have greater tolerance for errors than the Quadro cards

                                           

                                          You're saying that supposedly GeForce cards are more reliable than Quadros. So you're not sure about that?

                                          ECC ram is key when you have ram that has to refresh 24/7 for long periods of time. In those situations, an error can develop during a refresh cycle.

                                           

                                          Not sure what unrecoverable (and undetectable without ECC) errors have to do with refresh cycles. Could you point me to those tests?

                                          • 18. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                            ECBowen Most Valuable Participant

                                            Greater tolerance for errors means the Geforce cards will ignore more errors or certian errors more than Quadro cards but that has not been verified. I have simply been told that.

                                             

                                            Refresh cycles are what cause those undetectable errors. That is why I referenced that. You dont get paging errors from write and read cycles. As for the tests, I would have to go looking again to find them. Test your unbuffered ram for 2 weeks and come back and tell me how many errors you get. Last time I did the test myself the answer was 0.

                                             

                                            Eric

                                            ADK

                                            • 19. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                              Alex - DV411 Level 2

                                              Greater tolerance for errors means the Geforce cards will ignore more errors or certian errors more than Quadro cards but that has not been verified. I have simply been told that.

                                              If it can't detect them, it'll ignore them, passing errors to final renders. That makes sense. My deaf mouse is very tolerant to bad jazz. Or perhaps a better parallel: the chef at the corner coffee shop lost all sense of taste and can't tell chocolate from mouse droppings. Would you trust that chef with your Tiramisu?

                                              Refresh cycles are what cause those undetectable errors.

                                              Hardware problems (e.g. overheating), rust, shorts, bad chips, bit rot - don't cause errors? Only refresh cycles? Sorry if I am dense.

                                              Test your unbuffered ram for 2 weeks and come back and tell me how many errors you get. Last time I did the test myself the answer was 0.

                                              So GeForce cards are more tolerant to errors they can't detect? How would we know those errors even happen?

                                               

                                              Perhaps we should then test not one's particular non-ECC RAM (unbuffered does exist in ECC flavors) for two weeks but a couple thousand GeForce cards for a statistically significant length of time? Yet even that doesn't address the problem.

                                               

                                              The problem isn't that the errors rarely happen. The problem is: when they do, GeForce cards don't tell you, and you end up with hot pixels (at best) way into the last stages of your VFX pipeline and it'll cost the film some serious money in re-renders and delays to the production schedule. Whoever made the decision to use GeForce cards will likely have to kiss their career goodbye as well.

                                              • 20. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                                ECBowen Most Valuable Participant

                                                1. I was referencing random paging errors that are not related to hardware defect or failure. If there is a hardware defect then I want to know about it immediately which the client can visibly see if they show in render. Also this goes back to an old argument from many who actually understand how ram chips are constucted and graded, that ram companies have an excuse to use lower grade ram with ECC because they assume any errors will be corrected. Have you ever asked why some ram is half the price of other ram even though the profile specs are the same. This is not due to simply price margin difference. This is due to grade levels of chips selected when bidding on chips. The Grade 1A ram is the most expensive and has the highest density transistor count since they come from the center of the dye. Those also have the least likely chance to have any random paging anomalies. If this was any major issue anymore with ram chip refinement, manufacturers would add the 1 dollar cost to offer ECC on any unbuffered ram. They dont because they know it's not necessary anymore unless your running a server 24/7 for months. That is when you need that kind of ram. BTW problems such as overheating should never come into play unless the system was purchased from or built by inexperienced hands.

                                                 

                                                2. The person I spoke to who has contracted for Nvidia did not state the Geforce GPU's dont recognize these errors. What it means is they are programmed to ignore them as stated. These are the same physical GPU's as what are on the Quadro cards. The only differences here are firmware and drivers. This means capability is not the question. Programming is the question and as I said I still cant get that verified.

                                                 

                                                3. Test as much unbuffered ram as you want for 2 weeks. Gluck finding any showing random paging errors if they have any grade quality to them at all. Since you mean Geforce cards the point here is simple. Buy the Geforce cards with the Lifetime warranty and are not the cheapest. Why because they are buying higher grade memory chip parts. EVGA is a perfect example. Once again you get what you pay for.

                                                 

                                                4. Ok the timeframe statement along with re-renders is really overly done. 1. Film time frames are not something you really want to reference for this. They are often months in rework and re-editing not to mention re-shooting. Having to re-render a 15 min reel is nothing compared to the production side. I dont believe you see to many Red film guys up in arms in the forums because their Titan card layed a bad frame. I have not seen a single 1 yet actually. Feel free to go find 1. When your talking GPU rendering of CG frames then those are rendered as separate frame files. I really dont think having to re-render a few frames is going to end someones career as you state. However to make a point, I have many clients doing work with both rendering often daily with Geforce cards including Titans and I have yet to have 1 call me and state they had a random frame error from the GPU rendering. If this really is such a concern at the start then those clients wont even use GPU rendering because it's been shown CPU rendering is still more accurate over all. It's just considerably slower. For what ever reason that has become far less a concern to the time advantage.

                                                 

                                                As a side note, the field you should be referencing for accuracy and when it really does play a major role is engineering and mathematical calculations. Some GPU assisted calculations can take weeks or even months as you know. In that circumstance where data accuracy at the end is critical then the ECC and the Quadro precision really comes into play. I would not suggest otherwise then. No institute wants to release data public that turns out to be flawed due to processing. Nor would a Medical field client want to have their entire research wasted up to that point because of processing flaws over a long period of time. Engineering I think is obvious why that is so crucial. Who wants a building falling down on itself. Those are major reasons to use Quadro cards with those industries.

                                                 

                                                Another question. What happens to the HDD guy when he has a bad unrecoverable block where the film media is stored. How is this any different than the GPU side? Why didnt he use all SSD's for storage to eliminate the chances for this.

                                                 

                                                Eric

                                                ADK

                                                • 21. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                                  Alex - DV411 Level 2

                                                  It's amazing that with 20 years in IT and building systems, five years in college (computer science, applied math), a decent number of years on tech forums, and usually having a good time reading scientific research in tech disciplines, I barely understand half of what you're saying. Should I go back to school? Which one, Eric?

                                                   

                                                  Let's start with "paging" errors.

                                                  1. I was referencing random paging errors that are not related to hardware defect or failure.

                                                   

                                                  "Random" vs. what? Premeditated?

                                                   

                                                  Could you tell me what paging is doing here? (My understanding of paging: memory management scheme that has nothing to do with errors per se.)

                                                  3. Test as much unbuffered ram as you want for 2 weeks.

                                                   

                                                  No thank you. Google did that for me. This particular study places "annual incidence of uncorrectable errors" at "1.3% per machine and 0.22% per DIMM" on commodity servers. And again, "unbuffered" doesn't mean "non-ECC".

                                                   

                                                  Plus, if you're betting on your system being free of data corruption for only two weeks, I'd say it's a very, very unreliable system. I'd want my clients' systems to be free of data corruption or memory-related crashes for their lifetime (5+ years), and if that's not possible - for at least a year. There're plenty of other things that could crash a system, or cause data corruption, to also worry about undetectable memory errors.

                                                   

                                                  The same study: "8% of DIMMs in our fleet saw at least one correctable error per year."

                                                   

                                                  Those correctable errors would crash a system with non-ECC memory, or simply silently corrupt the data. Those happened in one out of 12 DIMMs once a year. Extrapolating this to our debate, a GeForce card would have an 8% chance of data corruption or crash a year. Quadro cards - 40 times less, a 0.22% chance.

                                                   

                                                  BTW, memory errors happen not only due to component failures. They also happen also due to radiation i.e. something beyond anyone's control.

                                                  • 22. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                                    ECBowen Most Valuable Participant

                                                    I was referring to page refreshes. Wrong term to use.

                                                     

                                                    Random versus consistent defect errors.

                                                     

                                                    I was referring to continued operation of the system without power down or restart to discharge and clear the ram. Yes 2 weeks of 24/7 use is good enough. At that point it should be shut down or restarted to discharge the ram.

                                                     

                                                    Yes 8% a year which is .02% a day.

                                                     

                                                    Random memory errors can happen due to signal attenuation on the bus between the controller and the modules. It's rare but I have seen it. Some board manufacturers would not support more than 4 Dimms on the X79 boards because of it nor would even some mount 4 slots.

                                                     

                                                    BTW not a single comment of mine suggested Unbuffered could not have ECC. The obvious point here would be turn it off for such a test. I used the term unbuffered because you cant get Non-ECC ram without it.

                                                     

                                                    Eric

                                                    ADK

                                                    • 23. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                                      Alex - DV411 Level 2

                                                      It's just considerably slower.

                                                      ECC memory considerably slower? How much slower?

                                                       

                                                      This 2006 test demonstrates no appreciable difference in speeds.

                                                      • 24. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                                        ECBowen Most Valuable Participant

                                                        Yes with DDR1 ram that had far lower latency numbers than the current ram. Neither was the amount of data pumping through the ram nearly as high then. We are talking 40+GB/s a sec now of data versus like 1 to 2GB/s then.

                                                         

                                                        Eric

                                                        ADK

                                                        • 25. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                                          Alex - DV411 Level 2

                                                          That doesn't answer the question, Eric. How much slower, on a decent system with a CPU that supports ECC?

                                                          • 26. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                                            ECBowen Most Valuable Participant

                                                            When I get more Registered Dimms in I will be happy to test and post for you. I am waiting on the next order since the current batch is in Dual Xeons that have to ship soon.

                                                             

                                                            Eric

                                                            ADK

                                                            • 27. Re: Have a Quadro k5000. Should I get another k5000 or the Tesla k20c?
                                                              Alex - DV411 Level 2

                                                              Thanks Eric. Until then, shall we refrain from calling ECC memory "slower", especially "considerably", given that several good resources point to 0-2% difference, perhaps closer to zero?

                                                               

                                                              "ECC may lower memory performance by around 2-3 percent on some systems, depending on application and implementation, due to the additional time needed for ECC memory controllers to perform error checking. However, modern systems integrate ECC testing into the CPU, generating no additional delay to memory accesses." (source)