1 person found this helpful
With that fast RAID 3 array I would personally keep everything except the paging file on the big array. Less housekeeping problems and it would not speed things up. That is what I do. Also I do not bother moving the cache files from the C: drive and avoid using them by using AME directly. If you export from Premiere then the cache files are used.
1 person found this helpful
SSD for scratch, cache and pagefile - quick answer is no!
You passed on using a SSD for the only decent fit for a CS5 rig, which is to use them for your OS/programs drive (or array).
I ran PPBM5 NUMEROUS times trying different drive combinations and permutations and the only thing that SSDs seemed to improve over vs. rotating drives was "media cache DB" (but no the "media cache"), and even that did not account to much difference at all.
Thank you both. So basically, a 9-disk (or 10) RAID3 is more then enough to run everything without adversly affecting the performance (except the pagefile).
Jim, having tested various permutations, if I used a fast SATA3 SSD for the boot and app drive, because of it's speed, I presume keeping the Pagefile on it would be the thing to do? I'd be currious to find out what you found to be the most optimal configuration (budget asside). And in your tests, have you tried RAIDing a large amound of SSDs, compared to similarily numberd HDDs in a same RAID level? I would expect a tremendous gain, but between theory and actual performance, there's usually a world of difference.
Also, have you tested the number of HDDs you can put in a RAID before it becomes inconsequential, speed-wise, to put more then X amount of disks? I'd be curious to max out the Areac to 16 drives. And at what quantity drives does it becomre interresting to use RAID 30 instead of RAID3? If I did max out the RAID card, to save space, would 2.5" HDDs be rugged enough to withstand the pushishment of a RAID setup?
Your questions regarding drive arrays and system setup are good, but if I or anyone were to answer them we would be influenced by our own test files, projects we have worked on, CPU, RAM, controller card (or lack of thereof), and more.
Since what you are building has more than a few similarities to my own, I will share some of my personal findings. Regarding specific answers like "how many drives in a RAID before it becomes inconsequential", that depends on so many things that you should probably stop asking questions like that, since it can depend on some many things - CODEC, number of layers, filters/transitions, etc.
Things I will comment on that are based on similarities of the system you are building and mine (X58, 6-core CPU, 24GB RAM, Areca controller w/ large cache (2GB or larger):
1. Moving the pagefile around really didn't seem to make much difference on my system. So as a result, I keep in on my OS/Programs array.
2. Fastest I've run was using two large arrays with projects, media, and all "scratch" files on one array, and "media cache" and output on the second. While the difference was not huge, the fastest result for PPBM5 benchmarking was achieved but putting the "media cache DB" on my OS/Programs array, which uses SSDs.
So how can you proceed?
1. Load your OS and programs onto your boot drive, or boot array.
2. Configure your Areca and drives in RAID 0, load up some actual projects similar to what you will be using this system for and test it out. Run PPBM5 too, it is a very useful common denominator that can be used to tweak your own system and to make before / after comparisons of changes that you try. IMO, a 9 or 10 drive RAID 3 (or 5 or 6 for that matter) will work pretty well for everything. [Readers here that have big RAID 0's off of their motherboard, I'm not sure if this is true for you or not; my testing has been done with all large arrays hanging off an Areca controller]
3. To answer your own question regarding "how many HDDs do you need" on your own system and with your own projects, simply redo step #2 using a smaller RAID 0 array. The latest Areca SAS controllers do RAID so fast, that I would say you can count on very comparable performance with a RAID 0 array with one less drive than when you build the system back with your chosen PARITY RAID; except for RAID 6 of course, which would effectively perform "2 drives" smaller.
4. Configure your final selected array(s) with parity, load your projects, and you are done!
Another way you could proceed? (Less time consuming, less optimized build, potentially case filled with more drives, heat and power consumption that the "optimal" build) Note that I have tested with some larger RAID 0 arrays than will be in my final build, and I use those drives for off-site backup storage.
1. Build your large array with PARITY from the start.
2. Start using the system and if it works OK for you, you are done.
Regarding RAID 3, 30, 5, and more, I use my data array for more than video editing, so going with RAID 5 was a simple choice for me. That being said, it seems to me that in my system (using a large cache ARECA controller and 1TB 7200 32MB cache drives) a "one less" drive RAID 0 seems to run just about the same as a RAID 5, so I don't think that the inefficiency of RAID 5 vs. RAID 3 for large files (aka video editing) could be that noticeable.
My editing machine has 8 each 2.5-inch Seagate Savvio 15K.2 SAS drives (ST9146852SS) in RAID 0 which I got a bargin prices. Sometime when I get enough time I plan on gradually removing them one at a time and see the PPBM5 results. Everyone keeps telling me I am crazy to keep them that way. But the only thing faster on PPBM5 Disk I/O testing is a 5.2 GHz i7-2600K (by the way Disk I/O test results are faster on the Sandy Beach Here are my read and write benchmarks.
Thank you so much. That's a real wealth of information. This answers a lot of my "question marks" I've had these past several days.
I've been sticking to 3.5" Hitachi 7K3000 for the RAID setup. But thinking ahead for future expansion, I'd like to have my options open, by using 2.5" HDDs (with 2x 2.5" in 1x 3.5" cages). Other then SAS drives, which are normaly well above the prices of any SATA drives, do you have any recommendations on models?
Bill, for curiosity's sake, do you know why the Sandy Bridge is faster in Disk I/O tests?
Also out of curiosity, is it possible to have SAS and SATA raids on the Areca cards; ie 4x150GB SAS RAID 5 + 4x3TB SATA RAID 5?
I think it was you that had mentioned going with Newegg's fabiously priced 7k3000 1.5TB drives. Totally awesome way to go in my opinion.
In another thread, I noticed that you incorrectly stated that the Hitachi drives you plan on buying are certified for use with Areca; that's not really true, but I'd still say no worries. Areca pretty much sticks to listing Enterprise drives for cerification with their RAID cards (i.e. Hitachi UltraStar, but not DeskStar, WD RE3/RE4, but not WD Black, and so on).
I've been using Areca controllers for a couple of years now and loving them; I had previously used 3WARE. I've used WD RE3, WD Blacks (w/ TLER patched), WD Blacks (newer drives from WD that will not accept the TLER patch), Hitachi 7k1000.C, and most recently Hitachi 7k3000 3TB drives - all glitch free, and all in both RAID 0 and RAID 5 configurations during this time. My own personal theory (unproven) is that Areca, and possibly other vendors too, have made smarter firmware that doesn't drop the consumer drives as quickly as they possibly did a couple of years ago. In my Areca configuration screen, for most of my drives "Error Recovery Control" shows to be disabled. While I'm just one user saying this, the geeks running 48TB and up video server systems are really putting consumer drives though their paces (ref hardforum.com) using the 24-channel Arecas, and are choosing certain brands and certain models since they have "proven" themselves over time.
Frederic, the only thing I can guess at so far is since the Sandy Bridge's have fewer PCIe lanes there may be less overhead serving the disks
Jim, I have actually done that a long time ago.
Also I have noticed that CS5.5 is faster on Disk I/O than CS5
Jim, Bill, many thanks to you both!