3 Replies Latest reply on Jul 24, 2011 3:25 AM by Harm Millaard

    PPBM5 Benchmark new results format

    JKad Level 1

      I have to say that I really like the new and much improved results page with sorting capability.  It sure does make it easy to sort on most any of the system parameters.  I know that this page took a LOT of work and I must say, Great work Bill and Harm (listed alphabetically).  Your continuing work is greatly appreciated.

      Some observations followed by a question:  As you've pointed out to me, the benchmark files currently used result in materially higher times for the MPEG2-DVD test for those of us with CS5.5 vs. the results the same computer would get with CS5.  I believe it was Bill who told me that this is a result one sees in the benchmark results but that on the same computer, CS5.5 is generally actually faster than CS5 with MPEG2-DVD conversions.  Now that we can sort the results, we can do a lot more comparisons.  But since the results of the MPEG2-DVD test are materially different for the two versions, the overall results are skewed thus making comparisons difficult.

      I'm sure you have plans to deal with this in some way.  And based on some other information I've seen, I believe your working on devising tests that will also provide better data on the impact of more/fewer CUDA cores.  I would greatly appreciate an update on your plans to deal with the CS5.5 vs. CS5 MPEG2-DVD issue and what we might hope to see at some point.

      After building a new computer, PPBM5 helped me identify what was working well or needed further tweaking.  It also gives me some idea of other hardware I could consider to improve performance. Thanks again for providing this extremely useful tool.  

        • 1. Re: PPBM5 Benchmark new results format
          Harm Millaard Level 7

          I've just finished further tests on the new pages, but using XML, instead of HTML. That allows me to now start working on dynamic Flash charts with drill down capabilities with Fusion Charts to further enhance readability and analysis possibilities. For instance when you select the Raid controller tab and hover the mouse over a hot-spot, a pie-chart will appear like a Spry tooltip with the distribution of the % of systems using 'None', 'On-board', 'Areca', 'LSI' or other controllers. Same for Manufacturers, Model CPU, # RAM, etc.

           

          The next step is to use CPU-Z to gather all the hardware information in Ghost mode, so you, the user will not notice it running, but attaching itself automatically to the submission form and preventing you from entering all the questions about your hardware. Once that is done, I can make pop-up windows with CPU-Z like information about the hardware, when you select a certain record, kind of like master/detail pages.

           

          Then the next step is to link all the hardware info from each system to external databases, and show up-to-date BFTB charts based on current prices.

           

          This is pretty ambitious for a single person, who is now taking his first steps in developing a website with mySQL, php, js, AJAX, Spry, XML. HTML, jquery and the like, it only takes far more time than I expected and hoped for.

           

          I make no promises that I will succeed, but my intentions are pretty well defined above.

           

          On the http://ppbm5.com/DB-PPBM5-2.php page you can select to narrow your search to CS5.5 only (currently 40 systems with BillG in top position at rank 13) if you want. Personally I like the query possibilities even more than the sorting capabilities.

          1 person found this helpful
          • 2. Re: PPBM5 Benchmark new results format
            JKad Level 1

            BillG's results point out the issue I was questioning.  Using 5.5 his system is ranked number 13.   Using 5,his system is ranked number 3.  The MPEG2-DVD results drop dramatically using 5.5.  Is there a fix in the works?

            Wow, you have a lot planned.  With all you have already done, the capabilities are really taking off!

            • 3. Re: PPBM5 Benchmark new results format
              Harm Millaard Level 7

              The problem is that in CS5.5 internal cache algorithms changed rather dramatically and what we never expected now suddenly is true. The design of the MPEG2-DVD test is nothing more than three instances of the H.264-BR sequence after each other. With CS5.0x and it's caching, you could see that the results were very good, but the changes in the caching in CS5.5 cause the much lesser results.

               

              If we were to modify the MPEG2-DVD timeline to avoid the replication and thus the caching issue, we would also invalidate the existing results. Rather a nasty dilemma. However, since we are aware of this limitation, we can take it into account for CS6.