You know it hasn't been released yet, right?
It seems that CS5.5 has been officially released now.
Has anyone successfully downloaded and tried the 30-day demo version with PPBM5?
And as asked in the OP, has anyone tried their system in a "before" and "after" PPBM5 benchmark results going from 5.0.3 to 5.5?
Downloading trial right now. I tried to get just Premiere but had to take the whole Production CS5.5 It is telling me about 3 hours
1 person found this helpful
Well I now have CS5.5 installed and running. I have made a PPBM5 run BUT, first of all things are moved around and our"Benchmark" file name is no longer carried through, you have to rename the files so the script can find the correct files. The biggest problem is that so far I have yet to find how to turn off the audio encoding which make the MPEG2-DVD and H.264 encoding time longer than before.
Back to exploring the new UI on AME to see if I can find how to turn off the audio encoding
I'm sure that is not what you or Harm wanted to see!
1 person found this helpful
Well I did figure out how to turn off the audio encoding. In AME 5.5 you select the sequence (Highlight it) and right click on it and a menu comes up with an "Export Settings" item. Selecting that then gives you the standard "Export Settings" window.
Now for the comparison and the good news first. My Disk I/O score dropped from 61 seconds to 55 seconds, my H.264 score dropped from 57 seconds to 43 seconds, and the Render Timeline with the MPE hardware acceleration stayed the same. Then there is unexplained bad news the MPEG2-DVD encoding went up from 21 seconds to 60 seconds! Of course this is all preliminary data until we get more experience, and do not forget our results are all +/-1 second because of the Windows one second clock resolution. So rather than a total of 144 seconds that I got with this specific current computer setup and 5.0.3, I got a total of 163 seconds.
Remember this is my first look at the new version and the first and only run of PPBM5 and maybe I need a little learning curve. I usually do not publish numbers untill I do a run of 10
With a brand new puppy in the house I probably will not get much more time at this for a while.
I'll try this asap and I let you know my reault.
Then there is unexplained bad news the MPEG2-DVD encoding went up from 21 seconds to 60 seconds!
That is surprising indeed! Is the resulting file size different by much?
Also, how hard were your CPUs working during the MPEG2 encoding (CS5 vs. CS5.5)?
PS - Great news about having a new puppy!
Last night I did not look at the CPU, but it is the highest GPU usage that I ever have seen for a very long time.
I did not check file size afterwards but the Export Settings window said the normal 93 MB.
Off to the dentist (wifey). This afternoon I will do some more testing.
I did a test with my current main system, and I can confirm that MPEG-2 DVD encodes are definitely a lot slower with CS5.5 than with CS5 5.0.3. In my case, it mushroomed from 30 seconds to 108 seconds. Total time in PPBM5 went up from 164 seconds to 229 seconds.
These results can mean any one of these things:
1) Premiere Pro CS5.5 (or rather AME CS5.5) works best with more than 24GB of RAM
2) CS5.5 works significantly better with hexa-core CPUs than with quad-core CPUs
3) The Intel Sandy Bridge platform is a bit overrated
I have yet to test my i7-920 with either 12GB or 16GB in CS5.5.
... or it could be:
4) a new MPEG2 encoder is used in CS5.5, and it is slower for some conversions than the encoder in CS5
Remember, when CS5 was released the trail version did not include the MPEG2 converter because according to Adobe, "The trail versions of Adobe... CS5... do not include some features that depend on software licensed from parties other than Adobe. For example, some codecs for encoding MPEG formats are available only with the full version of these products... Adobe Media Encoder export does not include MPEG-4, MPEG-2 DVD, or MPEG-2 Blu-ray as export formats in the trial."
Now, with CS5.5 just released, all converters have been included with the downloadable trail that is out.
Certainly Adobe still may be using third party licensed software in CS5.5 and could have just changed their agreements on what could ship with trail versions, but with the rather significant increase in MPE MPEG2 times seen by Bill and now Randall, it does make one wonder if the encoder itself is different on some more fundamental level.
I have seen the same deterioration in MPEG2, although I can't currently tell you by how much, but it was sizable enough for me to file a bug report.
I'll be back with some solid figures when I have the chance. AVI and H.264 show no significant change, but still the direct export of AVI is nearly four times faster than with the queue.
The problem is NOT CS5.5 (well at least not directly) it is the way we had had PPBM5 constructed. If you will take a clip of your own and MPEG2-DVD encode it, you will find CS5.5 is faster than CS5 here are some numbers of a 3 minute 1 second full AVCHD clip with color correction rendered four different ways with all the same presets on my fast machine
Direct Export Premiere 5.0.3 174 seconds
Via Sequence Import AME 5.0.1 142 seconds
Direct Export Premiere 5.5 103 Seconds
Via Sequence Import AME 5.5 96 seconds
So you see it is Premiere 5.5 that is faster. We believe we know what the problem is with PPBM5 but are not sure how to handle it without essentially invalidating all the current non-5.5 scores (In other words we will have to have a separate test and results page) Each time a new version of Premiere has come out in the past we have had to make changes and start all over .
Message was edited by: Bill Gehrke
Well you can see that Harm and I are not in complete sync or maybe there is a possible difference between quad-core (Harm) and hex-core (Bill) or who knows what?
Wow Bill, those are huge improvements!
Are you thinking now that the PPBM5 CS5.5 reported MPEG2 "slowness" is simply a calculation error in the VBS script?
No, there are no errors in the VBS script, because we have other means to calculate the time and these other methods corroborate the VBS results.
Initial thoughts, that have to be tested further, is that CS5.5 misses out on memory caching efficiency in comparison to CS5.03. We have previously seen the same happening with MPE efficiency and the way it caches its data.
Initial timings with 5.03 versus 5.5.
Source: Single clip AVCHD 1920 x 1080/25i with 5.1 sound, duration 35:26:22, no effects, no transitions.
Encoding to PAL DV AVI, default settings, MPEG2-DVD NTSC High Quality Widescreen i and H.264-BR HDTV 29.97i, all with PCM 48 KHz stereo.
DV PAL MPEG2-DVD H.264-BR Direct Export 5.03 8:59 Not tested Not tested Queue 5.03 15:03 21:52 42:55 Direct Export 5.5 8:53 Not tested Not tested Queue 5.5 8:45 10:34 43:09
This was only a single run, without optimized settings, while multi-tasking, but it shows that in this instance:
The severe discrepancy between direct export and queueing of DV material is now resolved.
H.264 export makes no difference at all. The differences are too small to be noticeable.
MPEG2 export has improved significantly on a single clip.
Direct export of the PPBM5 Disk I/O timeline uses - at least on my system - only 18 seconds and with the queue 64 seconds. This can only be explained by the caching efficiency of the DV export engine with direct export, lacking in the queue cache. This was the same with 5.03 and now still apllies to 5.5.
H.264-BR shows no discernible differences between versions.
MPEG2-DVD shows a marked improvement with a single clip, where caching is not relevant, but in 5.03 there was ostensibly some caching mechanism at work that recognized the three instances of the same sequence, leading to improved performance on the benchmark test. That caching mechanism seems to have been bypassed or disabled, resulting in PPBM5 results that are worse than in 5.03.
Caveat: Since this was done on a single run, with a number of background processes running, I suggest to disregard differences in the seconds because of statistical erros. IMO the H.264 results are for all practical purposes identical, the AVI results (with the exception of the queue 5.03 figures) are identical.
What does your study imply about the validity of the PPBM test across different types of machines (using the same version of PPro)? Is this "caching efficiency" the same for all machines, or is it a variable parameter not captured in your tables?
You predicted in a previous post that the PPBM test would be equally applicable for CS 5.03 and CS5.5. It would be important to have a test independent of the PPrpro version because improvements in CS5.5 might be differently apparent in differing hardware configurations (such as: no difference in results between GTX-470 and GTX580 in CS 5.03; but perhaps a difference in CS5.5.). The PPBM5 results that have been popping up suggest that the same computer does give different results for CS5.03 vs. CS5.50.
Furthermore, the uncovered discreptancies in results with individual clips vs. clips that can undergo behind-the-scene caching (as you report in your very useful study) suggest that perhaps the test needs to be changed so that we don't run into the problem.
I had tried to use the PPBM test to check out computers employing a Matrox box (MXO2 or RT.X2) and Matrox add-on software; I now wonder whether the eyebrow-raising increase in the disktest (DV PAL) results that several of us Matrox users have reported could also be due to some artifact of the PPBM5 test procedure, where this caching in the PPBM5 test perhaps does not take place with the Matrox system (just a guess of mine now).
I appreciate the effort you have put into this very useful benchmark program, and hope that the effort to bypass these uncovered issues does not cost you too much time.
The caching is not part of PPBM5, it is strictly an Adobe/Windows software situation and is not hardware related.
As neither Harm or I employ any hardware from Matrox or any other special editing hardware it is difficult to test that sort of thing so we can only report what we are provided..
As this purely volunteer effort we have to really struggle to keep up with technology, users reports and incomplete or incorrect data. PPBM was developed back in 2005 when I could not find any "Premiere" benchmark that tested multiple facets (disk, CPU and now GPU's) running under Adobe Premiere. Every new major version of Premiere that has come out has required changes. Also be aware that I believe Adobe is going to check on this anomaly that PPBM5 has found and who knows maybe the change will surface later.
I had the Matrox MX02 mini max on my system and ended up removing it as the machine was slow and unstable(PPBM5 622 sec) I installed a Blackmagic Ultrastudio Pro USB3 and the machine is rock solid (PPBM5 159 sec). My biggest problem was waiting for updates from Matrox when Adobe updated. I would not like to be in the shoes of someone who has a Matrox card and has just purchased CS5.5.
I am waiting to upgrade from CS4, but MX02 mini does not currently support CS 5.5 (until mid-June they say.) Did you return the MX02? I read that you can still capture and edit with it, but not monitor. Just don't use the Matrox sequence presets. I hope we don't have to wait much longer. If you hear anything different, please post it because Matrox is not saying much.
The current situation with drivers is discussed on the Matrox MXO2 board: you just have to wait a few weeks longer for the software update. That said, I have both the CS5.03 and Cs5.5 installed on the same machine, whith the appropriate Matrox driver installed on for Premiere 5.03. In your case, you could run a Matrox/CS4 partition, and a pure Adobe CS5.5 partition as well. Having the Matrox system on your computer won't hurt you with the CS5.5 system.
Frankly, I found the CS4 version too buggy for my taste (I am trying to be polite here). If you get CS5.5 now, you might need all the time you have until the release of the Matrox drivers to study the Adobe CS5.5 version anyway. I also would first make sure I had a top-noch supported video card (GTX-470 or better GTX-570), and then think about the additional advantages provided by the MXO2 card (a few) or an Intensity card (a slightly lesser few) in your specific work flow. But as a first step, get rid of CS4!