4 Replies Latest reply on May 15, 2014 4:46 PM by Bill Gehrke

    Testing 4x E5-4650s and 128GB RAM - tweaks for 32-core performance?

    NG Media Level 1

      Howdy. I've got the following system:

       

      4x E5-4650 eight core CPUs (32 cores/64 HTs total)

      16x 8GB EEC RAM - 4 sticks per CPU

      2x 512GB Samsung 840 Pro SSDs

      5x 3TB Hitachi HDDs in RAID5

      1x GTX 670 w/ 4GB RAM

      Windows Server 2012

       

      When I'm exporting 1080p from Premiere Pro CC with software encoding I'm getting really low CPU usage - around 4% usage, If I enable Mercury Engine it jumps to about 20%. Any tips or tricks?

       

      *EDIT: I'm exporting from various sources and frame rates to 1080p H.264@50fps I'm happy to run some tests.

        • 1. Re: Testing 4x E5-4650s and 128GB RAM - tweaks for 32-core performance?
          Bill Gehrke Most Valuable Participant

          Well there is only one current real Adobe Premiere Pro BenchMark and that is our PPBM6/7 benchmark.  If you download and run this four test benchmark we can advise you on the performance of your supercomputer.  It can tell us the performance of your disk system, the CPU and the GPU.  As you have so much CPU power and very good disk drives it appears to me that the weakest link in your system might be the GPU.  Unfortunately we have not yet seen usage of the Windows Server 2012 which is supposed to work and is required to use your 4 CPU's but is not supported by Adobe.  Our benchmark has been designed to stress the GPU, CPU, and the disk system.. You will have to register on the PPBM7 site to see the results, but I would strongly suspect that you can set new records.

          • 2. Re: Testing 4x E5-4650s and 128GB RAM - tweaks for 32-core performance?
            NG Media Level 1

            Just finished PPBM7 and got the following:

             

            Disk I/O Time = 244s

            Premiere disk write rate is 152.02MB/s

            H.264 Encoding = 126s

            MPE gain = 293/97 = 3

             

            The disk I/O time is strange as I get the following with CrystalDiskMark:

            2x SSD.PNG

             

            While doing the PPBM disk test this is the disk activity reported:

             

            During PPBM7 Disk test.PNG

             

            It seems to average out at about 500MB/s according to the resource monitor but only is shown as 152MB/s? It also has the OS on it so that might be an issue. Here are the ATTO Benchmarks, which seem pretty on the mark for the specs:

             

            ATTO 2xSSD.PNG

             

             

            I think there are definitely major changes needed to get better performance out of this system. I'm just running a few tests now and it seems that Windows 2012 is pretty shocking compared to WinSvr 2008: Supermicro X9QR7-TF+/X9QRi-F+ vs Supermicro X9QR7-TF+/X9QRi-F+ - Geekbench Browser

             

            Having a look at the benchmarks for the 11TB RAID5 shows some amazingly bad performance - 11MB sequential write, 200KB/s random 4K write. I get better write speeds on my USB stick. Either CrystalDiskMark is screwy or the raid setup is totally broken. It's setup using the onboard LSI 2208 controller, which I though was pretty decent. I'm guessing since my disk is 92% full that really messes with performance:

             

            5x 3TB RAID5.PNG

             

            ATTO gives clearer picture, with transfer sizes below 32KB being crippled to under 1MB/s:

             

            ATTO 5x3TB.PNG

             

            But of course when I copy a single 40GB file from the RAID0 SSDs to the RAID5 HDDs I get this, which I'm totally ok with:

             

            Copy from SSDs to HDDs.PNG

             

             

            Anyway, the major issues is really that the CPU is really underutilised and only uses about 30%. It's over 80% on one of the NUMA Cores but only 25% on the others. Looks like this:

             

            NUMA Cores.PNG

             

            I suspect that Premiere Pro is only detecting 32 of the threads, not 64. In the Multiprocessing preferences in After Effects it only shows up as have 32 logical processors, not 64. Thinking I'm having the same issues as this guy (same mobo and processor): AnandTech | Trials of an Intel Quad Processor System: 4x E5-4650L from SuperMicro

             

            Any help would be appreciated.

            • 3. Re: Testing 4x E5-4650s and 128GB RAM - tweaks for 32-core performance?
              Bill Gehrke Most Valuable Participant

              I only have a few minutes right now but those are disappointing results.  Your disk system is set up terrible for Premiere.  Your boot disk a pair of 512 GB SSD's is not practical  Have you turned off indexing on all drives?  Look at Harm's Tweakers Page under both disk headings.  Have you registered on PPBM7 so you can view the Results page?  Have you tuned the system?  I would suggest you get another smaller SSD and use it for you boot and application disk and not waste those two 512 GB SSD drives

               

              I will be back online in about 7 hours.  I really have not had a chance to read your whole posting Maybe some one else will check in in during the meantime

              • 4. Re: Testing 4x E5-4650s and 128GB RAM - tweaks for 32-core performance?
                Bill Gehrke Most Valuable Participant

                Well guess what Adobe has finally solved the sign-in problem on the forums after 30+hours--I hope

                Here is the the response that I prepared over 24 hours ago but could not post.

                In my absence no one could check in due to a serious Adobe problem with User ID's

                1. Make sure you have indexing turned off on all drives.

                2. A huge mistake is trying to operate with 92% full drives.  It is alright to have a large archive but not for a current project.  A general rule-of-thumb is never have a project disk over 65-70% full

                3. You should have only one smaller SSD for your boot application drive, adding the second SSD will not improve editing performance.

                4. I would use your two 512 SSD GB that you have in RAID 0 for your current projects and provide a goobackup plan.

                5. You should probably get yourself several new hard drives and use them as suggested on the Tweakers Page

                6. Your current RAID 5 array is probably fine for your current archive, just do not add any more to it or try to using it with current projects.  I suspect you have a lagging disk causing that write problem.   Those older Hitachi drives are not enterprise drives.  Now I have used non-enterprise drives very successfully, but I test each drive separately with HD Tune Pro especially for write operations, I have found drives this way that are not suitable for RAID operation.  RAID 5 is not a great type of RAID for big Premiere files.  Of course that is the only Parity RAID available with that LSI controller.

                7. After fixing the disk system with this fantastic system you may have to get an expert with real knowledge of your Windows Server 2012.  Essentially no one on the forum has ever used this OS.  There must be dozens of features, services and processes that can be turned off when using it as a workstation.  Harm’s pages have a tuning guide but this is only for Windows 7 or 8.

                8. After this is done you probably want to consider a new and/or second GPU card.