15 Replies Latest reply on Jul 5, 2010 5:06 AM by Powered by Design

    Error when running PPBM4 BENCHMARK test

    rowby Level 1

      Please check out the problem in the following short video:

       

      http://www.youtube.com/watch?v=OUeBdg6oFBQ

       

      Thanks

       

      Rowby

        • 1. Re: Error when running PPBM4 BENCHMARK test
          Harm Millaard Level 7

          Rowby,

           

          In Explorer and in the directory where the Statistics.vbs file is right click on the address bar in the top and select Copy address.

           

          Then double click on Statistics.vbs to start the script and on both the first and second question, use CTRL-V to paste the address.

           

          17-5-2010 15-19-28.jpg

          • 2. Re: Error when running PPBM4 BENCHMARK test
            rowby Level 1

            Hi

             

            You will see in my new YouTube video my attempt to copy the address from the Windoes Explorer address bar and paste it into the prompt.

             

            Getting same error.

             

            http://www.youtube.com/watch?v=OBV69If_xxc

             

            I tried every variation I could think of.  One minor issue is cs5 needs to convert the cs4 file to cs5 -- and when it does so, it renames it to Benchmark_1.prproj.  No big deal, I just do a save as to save it to Benchmark.prproj.  But it would help me troubleshoot the problem if the project was already in CS5 because I keep thinking that maybe the problem has something to do with the conversion.  I doubt it, but you never know when troubleshooting....  Also the first couple of times I tried to get it to run I didn't pay attention to the automatic renaming of the project, and when I ran the script of course it could not find the resulting files because they had the appended number on them.

             

            I tried using different folders without any luck, so I put the contents of the zip file into the very root of my "E: directory.  So the rest of this post is based on my project and everything else in the E: root directory.

             

            Also,  for some reason, even if I tell the preview files to be in the same folder as the project, Premiere CS5 appears to create the Adobe Premiere Pro Preview Files folder anyways and puts the preview files in that folder, not in the E\ root directory.  (My eyes and brain are blurry at this point so I may be mistaken about that....)

             

            And I see the warning "DO NOT enter \Adobe Premiere Pro Preview Files\Benchmark.PRV"  However what *does* it want me to put in there:   I assume (in my case)  I need to enter: "E: \Adobe Premiere Pro Preview Files\"   Or should it be "E: \Adobe Premiere Pro Preview Files"  (no trailing "\")  ?????

             

            No matter what I try I either get a file not found or a overall error.

             

            A suggestion:

             

            Can the script be re-written so that the project MUST be in a folder off the root named PPBM4 and the preview files be put in the usual Adobe Premiere Pro Preview Files folder, since apparently that's what Premiere CS5 will do anyway (at least I could not find a way to put those files in the same folder as the project files, namely "E:\"

             

            That way the only input the person would have to do, to get the script working is to put the Drive letter and nothing else.  Not E: not E:\ Just E.  (or D, or F or whatever folder that person is using for their PPBM4 test.  In other words, dumb-downing the input required to get the script running.

             

            Thanks!   And of course I absolutely appreciate and thank Bill for creating this benchmark system. I hope my suggestions are helpful and perhaps easy to implement.

             

            Rowby

            • 4. Re: Error when running PPBM4 BENCHMARK test
              rowby Level 1

              Hi Harm,

               

              I double checked the article, and yes, I have followed those steps, doing the file save as Benchmark.prproj.   But still getting the error.

               

              What did work for me was to copy the preview files into the "E" root directory -- only then did the program find the preview files.

               

              Doing my benchmark now.

              • 5. Re: Error when running PPBM4 BENCHMARK test
                rowby Level 1

                BTW, does Sun Java need to be installed on a computer to run the benchmark script?

                 

                Rowby

                • 6. Re: Error when running PPBM4 BENCHMARK test
                  Powered by Design Level 4

                  I just finished running the PPBM4 test on my new computer.

                   

                  Its just an off the shelf from Best Buy but it beats my old Dell XPS at a score of 329.6

                   

                  New score is 60.9

                   

                  Here is my new scores.

                   

                  ....................................................

                   

                  Powered by DESIGN,   Personal or Computer ID

                  Gateway,  Computer Manufacturer

                  FX6840,   Computer Model

                  60.9,  secs Total Benchmark Time

                  4.1,  secs AVI  Encoding Time

                  35.8,  secs MPEG Elapsed Time

                  21,  secs Rendering Time

                  INTEL,    CPU Manufacturer

                  I7-860,    CPU Model

                  2.8 GHz,    GHz CPU speed

                  1,    Number of CPU chips

                  4,    Total Number of Cores

                  8 GB,    GB RAM

                  4.2.1,    APP Version PPBM4 DV

                  Win 7  64  Home,    OSVersion

                  SATA,    OS Disk Interface

                  1 TB,    GB OS Disk Capacity

                  7200,    OS Disk Speed

                  SATA,    Project Disk Interface

                  1 TB,    GB Project Disk Capacity

                  7200,    Project Disk Speed

                  SATA,    Preview Disk Interface

                  1 TB,    GB Preview Disk Capacity

                  7200,    Preview Disk Speed

                  SATA,    Output Disk Interface

                  1 TB,    GB Output Disk Capacity

                  7200,    Output Disk Speed

                  Radeon HD 5770,    Graphics Board

                  Added hard drives and Firewire card,    Comment

                   

                  ..................................................

                   

                  Post your output here too.

                   

                  I would like to see how your new build is doing.

                   

                   

                  ENjoy:  GLenn

                  • 7. Re: Error when running PPBM4 BENCHMARK test
                    Harm Millaard Level 7

                    Glenn, nice improvement! Imagine what your score would be with CS5, probably below 40 s.

                     

                    Did you mail these results to Bill, or will I add them to the results? No sense in doing it twice.

                    • 8. Re: Error when running PPBM4 BENCHMARK test
                      Bill Gehrke Adobe Community Professional & MVP

                      Harm the results are posted along with Rowby's and several others

                       

                      Rowby I am missing one piece of information on your configuration, what RAID level is your project array?

                      • 9. Re: Error when running PPBM4 BENCHMARK test
                        Powered by Design Level 4

                        I just finished running the PPBM4 test on CS5

                         

                        My times are Slower ?

                         

                        I will forward these to Bill again.

                         

                        ........................................

                         

                        Powered by DESIGN,   Personal or Computer ID

                        Gateway,  Computer Manufacturer

                        FX6840,   Computer Model

                        63.1,  secs Total Benchmark Time

                        7.3,  secs AVI  Encoding Time

                        32.8,  secs MPEG Elapsed Time

                        23,  secs Rendering Time

                        INTEL,    CPU Manufacturer

                        i7-860,    CPU Model

                        2.8 GHz,    GHz CPU speed

                        1,    Number of CPU chips

                        4,    Total Number of Cores

                        8 GB,    GB RAM

                        CS5  5.0.1,    APP Version PPBM4 DV

                        Win 7 - Home 64,    OSVersion

                        SATA,    OS Disk Interface

                        1 TB,    GB OS Disk Capacity

                        7200,    OS Disk Speed

                        SATA,    Project Disk Interface

                        1 TB,    GB Project Disk Capacity

                        7200,    Project Disk Speed

                        SATA,    Preview Disk Interface

                        1 TB,    GB Preview Disk Capacity

                        7200,    Preview Disk Speed

                        SATA,    Output Disk Interface

                        1 TB,    GB Output Disk Capacity

                        7200,    Output Disk Speed

                        Radeon HD 5770,    Graphics Board

                        Stock Best Buy model then added extra hard drives and firewire card.,    Comment

                         

                        Still like CS5 so far though.

                         

                        I dont have an approved CUDA card but it is working fine for this computer anyway.

                         

                         

                        GLenn

                        • 10. Re: Error when running PPBM4 BENCHMARK test
                          rowby Level 1

                          Hi Bill,

                           

                          It's a Raid3

                          • 11. Re: Error when running PPBM4 BENCHMARK test
                            Bill Gehrke Adobe Community Professional & MVP

                            I do not understand why you are not getting much better MPEG encoding results with CS5 versus CS4.Harm and I (probably others) on the CPU intensive MPEG encoding score get an improvement with a change from CS4 to CS5 that drops the MPEG encoding time to ~25% of the CS4 time.as you can see in the PPBM4 results page.

                             

                            I am not familiar with the i7-860 architecture except to know that it has fewer pins, and less expensive chipset.  It uses a P55 chipset versus the X58 chipset, but maybe there is some feature in the X58 architecture that is not available in the P55 that Adobe was able to use to get such a immense improvement in the MPEG encoding.  It will be interesting when someone else with a i7-860 gets CS5 and runs the test for us.  Or we find the right person at Adobe that can comment on this situation

                            • 12. Re: Error when running PPBM4 BENCHMARK test
                              Powered by Design Level 4

                              I was surprized myself.

                               

                              Maybe its the board like you say being a P55 and not an X58

                               

                              I only got this system to run for e-mail and web and other stuff and will build a editing system later.

                               

                               

                              How is the CS5 test coming along ?

                               

                              Any timeline yet ?

                               

                               

                               

                              Thanks:  GLenn

                              • 13. Re: Error when running PPBM4 BENCHMARK test
                                Harm Millaard Level 7

                                Glenn,

                                 

                                Without MPE, comparing CS4 and CS5 results, shows:

                                 

                                AVI from 4.1 to 7.3 which is behavior that all CS5 users have encountered. Adobe has that bug replicated and it is logged as "To fix". Notice the results from CS5 users and CS4 users on the AVI test, just by looking at the colors.

                                 

                                MPEG from 35.8 to 32.8 which is disappointing. I would have expected something around 12 - 15 s. I like to ask you to rerun the MPEG test again, checking that all the three tick boxes at the bottom of the settings screen are unticked. Again if you look at the colors for MPEG encoding, all CS5 users do a lot better than CS4.

                                 

                                Rendering from 21 to 23 s, which I attribute for a large part to measurement error, because accuracy is in integer seconds, not in decimals, so your results may have been from 21.48 to 22.52, which shows up as 21 and 23. BTW, that is one of the complicating factors for CS5, to reduce measurement errors in the new benchmark. I have the feeling that CS5 is slightly slower than CS4 on the render test. In my case without MPE it went from 12 to 14.

                                 

                                It won't be long anymore, but one of the things we are hoping for is the solution to the bug. Otherwise interpreting results before/after the fix will be more difficult and can cause confusion.

                                • 14. Re: Error when running PPBM4 BENCHMARK test
                                  Powered by Design Level 4

                                  I will re-run the test but I know for sure I had all 3 boxes UNchecked

                                   

                                  GLenn

                                  • 15. Re: Error when running PPBM4 BENCHMARK test
                                    Powered by Design Level 4

                                    I ran the test 2 more times.

                                     

                                    I forgot to have my scratch files go to a different drive.

                                     

                                    I ran the test again the same as I ran the other 2 tests with all the files on ONE drive and here are the results.

                                     

                                    ...........................................................

                                     

                                    Powered by DESIGN,   Personal or Computer ID

                                    Gateway,  Computer Manufacturer

                                    FX6840,   Computer Model

                                    62.5,  secs Total Benchmark Time

                                    7.3,  secs AVI  Encoding Time

                                    32.2,  secs MPEG Elapsed Time

                                    23,  secs Rendering Time

                                    INTEL,    CPU Manufacturer

                                    i7-860,    CPU Model

                                    2.80 GHz,    GHz CPU speed

                                    1,    Number of CPU chips

                                    4,    Total Number of Cores

                                    8 GB,    GB RAM

                                    5.0.1,    APP Version PPBM4 DV

                                    Win 7 64 Home,    OSVersion

                                    SATA,    OS Disk Interface

                                    1 TB,    GB OS Disk Capacity

                                    7200,    OS Disk Speed

                                    SATA,    Project Disk Interface

                                    1 TB,    GB Project Disk Capacity

                                    7200,    Project Disk Speed

                                    SATA,    Preview Disk Interface

                                    1 TB,    GB Preview Disk Capacity

                                    7200,    Preview Disk Speed

                                    SATA,    Output Disk Interface

                                    1 TB,    GB Output Disk Capacity

                                    7200,    Output Disk Speed

                                    Radeon HD 5770,    Graphics Board

                                    Best Buy Special,    Comment

                                    .......................................................

                                     

                                    I then went in and changed the Scratch Disks from SAME AS PROJECT on all 4 options to a different hard drive.

                                     

                                    Captured Video

                                    Captured Audio

                                    Video Previews

                                    Audio Previews

                                     

                                    Here are the scores with that changes.   It got worse.

                                     

                                    ...........................................................

                                     

                                    Powered by DESIGN,   Personal or Computer ID

                                    Gateway,  Computer Manufacturer

                                    FX6840,   Computer Model

                                    87.4,  secs Total Benchmark Time

                                    30.6,  secs AVI  Encoding Time

                                    33.8,  secs MPEG Elapsed Time

                                    23,  secs Rendering Time

                                    INTEL,    CPU Manufacturer

                                    i7 860,    CPU Model

                                    2.80,    GHz CPU speed

                                    1,    Number of CPU chips

                                    4,    Total Number of Cores

                                    8,    GB RAM

                                    5.0.1,    APP Version PPBM4 DV

                                    Win 7 64,    OSVersion

                                    SATA,    OS Disk Interface

                                    1 TB,    GB OS Disk Capacity

                                    7200,    OS Disk Speed

                                    SATA,    Project Disk Interface

                                    1 TB,    GB Project Disk Capacity

                                    7200,    Project Disk Speed

                                    SATA,    Preview Disk Interface

                                    1 TB,    GB Preview Disk Capacity

                                    7200,    Preview Disk Speed

                                    SATA,    Output Disk Interface

                                    1 TB,    GB Output Disk Capacity

                                    7200,    Output Disk Speed

                                    Radeon HD 5770,    Graphics Board

                                    Best Buy Special,    Comment

                                     

                                    .........................................

                                     

                                    I thought it would be faster ?

                                     

                                     

                                    GLenn