OSX ability or lack thereof to corectly mulitthread is indeed part of the issue. and cross platform software audio or video runs better on windows.. this is just a fact.
on the other hand adobe can certainly use all the cores on windows. where the deminishing returns are with concern to cores is
6 core higher GHz vs 12 core lower ghz.. the higher GHz will always win.. with a few excepetions.
this is due to GHz wining not Adobes ability to corectly multithread.
our dual Xeon over clock to 4GHz was or still is the highest in PPBM
There are several factors involved with export times in AE. The first is the amount of ram handling the caching/buffering for all of the data on export and this changes based on the codecs used in the comp as well as the effects. What codec/format you export to will also change this. If you export to a Quicktime based codec then all observations I have done on Mac show poor threading overall. Some of the effects used also do not thread well. When combined you will see the export on the Mac seem to hit a wall. The larger the comp, the worse this is. This has very little to do with the system as a Mac Pro and everything to do with the player and caching of the player. Unfortunately there is not much of a resolution here that I have seen. This occurs on a PC system with AE as it does on a Mac but far less often. Almost all of the codec Options in Adobe on the Mac if not all are Quicktime based that I can remember. My suggestion is try and limit the scope of your comps into smaller segments if this becomes to much of an issue. Also make sure you dont allocate more than 1.5GB of ram per core in the multiprocessing options or the efficiency of Adobe's caching seems to get far worse. When you have time, I would run small comps through on export to see which codecs give you better performance. However keep in mind the overall size of the comp will really slow the export times down as it increases in scope.
I have tested OSX many times with different applications including Adobe and Final Cut studio and I have never seen OSX thread out to the Hyperthreading cores on anything accept drive instructions. Even then it only threads to 40% or so.
Hi Scott (and Eric) of SDK
I've seen this assertion " OSX ability or lack thereof to corectly mulitthread" before on forums related to Adobe CS. I guess I'd like to know if you think this is really a limitation of the Mac OS or of certain applications (specifically Adobe) that because they are cross-platform, have used more generic threading APIs than APIs that might take advantage of the Mac OS better. Other then you experimenting with certain apps, such as Final Cut Studio (I assume FCP 7?) and CS 5+ is there any other documentation or reports on the web you can point me to that Mac OS is just bad at mutlithreading? I have seen reports that FCP X can 'pin' all the processors in a 12 core Mac just fine, but with Premiere pro they get to maybe 50% max CPU usage for all the real and hyperthreaded cores. I have also seen multi-threading benchmarking applications 'pin' all the processors on Mac OS as well, so I suspect it's not an OS limitation, but a lack of optimization in the application.
What I really suspect is the culprit is that Adobe's foundation code base is built on Windows, and they have certain layers that deal with the specific OS for functions such as Multi threading. Because their interface is more generic and optimized to Windows and is built on more years and experience on Windows, it makes some sense that it would be less efficient on Mac. And if this is so, I think Adobe would be well-served to devote some resources to optimizing this part of their code to make Mac as efficient. Like it or not, they are going to have more and more Mac customers trying to come to Adobe, who are unwilling to switch platforms to PC for various and understandable reasons. (I'm one of those people.)
Anyway, thank you both for your helpful input, you are a great asset to this forum.
The Multithreading issues on Mac systems are really a combination of factors. With regards to videoediting/media content creation specifically, most of the threading issues revolve around Quicktime. Quicktime, atleast the SDK versions applications use, have very poor threading performance along with the 32bit limitation for ram. This is far more prominent on the Mac than the PC because Quicktime is used almost exclusively on the Mac.
The Multithreading issues in general though show across applications including video, audio, and other performance based applications. Pro Audio Applications such as Nuendo and Cubase thread very poorly on the Mac when they thread perfectly on the PC. This is really where OSX really begins to show it's limitations. There was quite an uproar back when the CPU's started reaching 6 Cores that the BSD Kernel had a major inherent limitation as designed. The kernel could only thread to 6 cores according to many. I personally have no idea if that is the case since I am not a OSX Developer. What I can verify is the threading is far worse the more cores you have. Hyperthreading has been unused by any applications that I have seen outside of drive testing/benchmarks. This so far has had nothing to do with certain applications whether audio or video specific. According to published information OSX has Grand Central Dispatch to handle moderation of threading for OSX and to improve multithreading performance. How that is actually implemented though and whether application designers have to build specifically for that I dont know. So far as current applications are showing, it is either not the most efficient process or it has limitations. If I had to hazard a guess, I would say it was implemented as a work around to inherent limitations of the kernel communicating with the Hardware. The reason I say that is because many who currently use a Hackintosh system report exceptional performance threading across all physical and Hyperthreading cores regardless of the core count or applications. This would indicate that the EFI Apple uses with the Kernel is causing the poor threading results and not the applications. Once again though I don't know for sure. I am simply reporting all of the considerable observations we have done. I sincerely doubt that that the performance issues are due to Adobe and the API they used for OSX. The poor threading is seen across all the applications we deal with icluding FCP 7 not taking into account reports by others the same is seen with applications we dont deal with.
I thought I'd provide a little update on the ability or inability of Mac Pro 12 Cores to fully multithread. Today, I was having a very hard time getting a sequence to render from PPro CS 5.5.2 either as a direct export or through Media Encoder. I have a feeling it was the Adobe QT Server which was hanging. When either hangs, I find this is the culprit. Usually a restart of the Mac solves it, until the next time. I find the usually when this hang occurs, Media Encoder is dead. Sometimes force quitting Adobe Quicktime server will help, but usually a reboot brings it back. I find Media Encoder pretty darn unreliable. It hangs about 30%-50% of the time for me. Probably becaue of the QT bugginess, though I'm not blaming Apple necessarily as it is an Adobe component that is hanging.
Anyway, I had this sequence export to h.264 which was just hanging about 3/4 of the way through, it was very short - less than 10 min, 2 tracks, 4 clips and simple, but did have some QT wrapped H.264 files in it. I did the usuall reboot, tried, Premiere Pro direct export, still hung. Out of desparation (I really needed to output this sequence) I turned off MPE CUDA to Software only. (I have a Quadro 4000 for CUDA)
I then did an export using Premiere Pro directly (not queued to AME). For the first time using Premiere Pro I saw all 24 logical cores on my 12-core Mac Pro pinned to near 100%. usually the most I get out of it is 40%-50% during a render. And the render was quite fast, about as fast as with CUDA on, I think.
Anyway, can you explain why my 12-Core Mac Pro has the ability to multithread properly when CUDA is off, despite what you and Scott have been saying about Mac OS's deficient multithreading? I'm not trying to be argumentative, but I saw this with my own eyes in Premiere Pro! Again, thanks to you and Scott for all your great info.
The reason you see 100% CPU utilization is because you have turned off GPU acceleration this is no difference in what I see when doing the same thing. 30 -40 % CPU usage with MPE GPU and 100 % CPU usage without GPU. That is my experience with a heavily loaded GPU accelerated effects and features with a MPEG2-DVD export.. But in using direct export from the timeline here is the encoding time acceleration that using the GPU offers over using the CPU alone
Thanks for the input Bill, I assume you are using Windows PC? I think my point was that it seemed that some folks were saying they never saw a Mac Pro 12 core get to 100% because the Mac OS was deficient and wasn't able to use multithreading as well as Windows PC. My point is that, at least in this instance, Premiere Pro as able to use all the cores properly. I do realize the benefits of GPU accelleration, I was just trying to illustrate that, at least on occasion the Mac can do this as well. By the way, I'm getting what I perceive to be much snappier performance without GPU MPE on right now during timeline editing. I think there is some bug or bottleneck going on. CUDA MPE should NEVER make things less snappy, don't you agree? I'm sure when I start adding CUDA effects all this snappiness and software MPE benefits will vanish. But for now with my simple timeline in the current project it's snappier.
Hmm actually looking back at my original post when I did the benchmarks before the Quadro cards were out for Mac, it looks like the same was the case then with Premiere. That would have been with Snow Leopard and CS5.0.2. When the Quadro cards released we never really tested again with Software MPE mode since Hardware was available. Times like these I wish Apple had not dumped so many of their resellers because I don't have MAC systems to test currently.