Why would you waste all that money on Velociraptors when (correct me if I am wrong) but the WD 2TB Blacks are faster and much cheaper?
WD Blacks are 7,200RPM, Raptors are 10,000RPM. But also, Blacks are not always a good RAID choice, they have time-out circuitry that corrects for erros when standalone, but can cause RAID to drop. Hence, the WD RE3 drives, made for RAID, no timeout circuitry.
Thanks, Harm, for your expertise! I've read this post and your post on general storage setup for editing systems, and I've got some questions...
First, I've got a TON of data, much of which I could probably archive to clean things up. Here's what I've got right now, and maybe you can give me some recommendations for fixing things so it'll run better (my PPBM5 shows significant slowdown in disc-land). FWIW, all internal drives are 7200RPM SATA2.
I have a 250GB WD Enterprise Storage drive for my OS/Programs, and I think it holds the pagefile... (though I have 12GB of Ram, so I'm not sure how often it's used). This one is about 50% full.
I have a 500GB WD Enterprise Storage drive which holds all my graphics, photos, and audio, along with some other generic data files (think documents, PDFs, etc. here) This one is about 60% full.
I have a 1.0 TB WD Caviar Black for my Projects/Scratch/Previews disc. This one is about 40% full.
I have a 2TB RAID-0 (made from 2x 1.0TB WD Enterprise Storage drives) that holds all my data in whatever format it came in on (most of it is direct capture from HDV in Premiere, but there's a good mix of AVCHD, R3D, and various SD stuff, too). This one is about 95% full.
I also have a 4TB RAID-0 WD backup drive that's a little over 50% full, and maintains a complete backup of all the drives in my system, minus OS and Programs. (FW800)
Then I have a second 2TB single disc drive that I use to maintain an off-site backup of whatever projects I'm currently working on, plus important stuff like photos and graphics. This one is about 30% full. (FW800)
Then I have a third 1TB single disc drive that is a general backup drive and usually remains onsite, and is about 80% full. (USB 2.0).
Ok, so there's where things are right now. Based on what I've read, I'm thinking that I might be better off doing the following:
250GB for OS/Programs
500GB for Previews/Scratch/Pagefile
3x 1.0TB drives (no RAID) for my projects/media OR 1.0TB in RAID3/5 for projects/media (but I'll have to figure a way to archive a bunch of stuff... Bluray?)
I've got room in my system to add one more drive, and I can probably get approval for purchase of one more drive if I asked my boss--so if I got a 2TB drive, could I do a 2.0TB RAID using two of the 1.0TB drives for striping and the 2TB for parity, leaving me with an extra 1.0TB? Or should I get a 1TB drive and do a 2.0TB Raid with two drives for striping and two for parity? I'm really not up on this RAID stuff, despite reading your article on it and Wiki...
Also, could I separate my current RAID-0 and not lose the data on it?
And just FYI, I'm using onboard RAID (using the ASUS P6T6 Server mobo), and purchase of a RAID controller is out of the question right now...
I primarily edit HDV/AVCHD, but I have a little bit of RED 4k and I'm hoping to upgrade my HDV camera to the JVC GY-HM700 soon.
Thanks for your advice and expertise!
I'm not sure I understand your recommendations for four disks. "D: or E: and F: can be run in Raid0". Does that mean D: and F: or E: and F:? Or does it mean D: or E: and F:? That's not the same. Hope you understand my confusion.
I run automatically in this four disk configuration because the SSD I would like to buy (C300) has excellent reads but slow writes. So it is good for OS/Programs and bad for the page file.
The first one:
Does that mean D: and F: or E: and F:?
Sorry for not being clearer.
I still don't understand
I believe he is saying
D: and F:
1. D & F
2. E & F
Thanks. So you can actuallysplit this 4-disk-option in two options: 4a) C:, D:, E/F combined; 4b) C:, E:, D/F combined.
E: and F: seems to make more sense though.
From your table I conclude on of the most important Disk Setup Laws: to separate the pagefile from media and projects.
Very nice chart!
I'm hoping you can provide some specific guidance for my situation.
I edit a lot of AVCHD and some MJPEG (all from DSLRs, 720p or 1080p).
I currently have two drives:
C: (Seagate 7200 RPM, 750GB) I use this for OS, Programs, Pagefile, Projects, Previews, Exports. I am not sure what Media Cache is, but I'm sure it's here as well.
D: (Western Digitial 2TB Green, slower than 7200RPM for sure, feels super slow to me, but haven't benchmarked). I use this just for raw footage exclusively.
Would you set this up any differently?
If I were to add one more drive, what makes the most sense? SSD for the OS/Programs/Pagefile? Should I just move the slow 2TB Green into an external enclosure and use for backups?
Message was edited by: Another Photographer
Get two new 7200 RPM SATA disks. Your current drive C and the new disks as D and E and use your current Green disk as F for backups. Then apply the three disk situation, C, D and E. You could consider using the F drive for exports with its huge size as well.
Thank you Harm.
In the 3-disk scenario, what should be the size of E:?
SATA disk start to slow down when you reach a fill rate of around 65 - 70 %, so if you expect to have say 400 GB of data, the size should be 640 Gb, if you expect around 600 GB of data, get a 1 TB drive, if you expect 1.3 TB of space get a 2 TB drive, if you need more use two disks in a raid0.
Thanks Harm. I was wondering how large "Previews and Media Cache" would be. My exports won't be huge, so if Previews and Media Cache were not large either, drive E: in the 3-drive setup would be relatively small (maybe 500GB?).
Get the largest you can get. A 500 GB is almost as expensive as a 1 TB. The difference in price is negligent, especially in comparison to CS5.
Thank you for all your posts. I have learned a lot by reading them. I am in the process of setting up a system for video production with CS5. I plan to purchase 4-10k RPM disks and had originally thought to set them up in a RAID 1+0 but I notice that you say repeatedly not to set up a RAID array with less than 5 disks. I am concerned that adobe requirements for CS5 specifically say that RAID0 is required for uncompressed video. Is that not the case and with that in mind, do you recommend that I set them up as single drives? Thanks.
A normal data rate for compressed 4:2:0 HD material is somewhere in the range of 25 Mbps, easy to handle for modern disks. Even with 2 or 3 video tracks. But uncompressed HD-SDI ingest in 4:2:2 HD format may go up to 1.485 Gbps, which means a sustained transfer rate of more than 200 MB/s per video track. In practice that would require a 3 or 4 disk raid0, depending on the fill rate. Imagine what would happen if you have four video tracks, that can add up to a sustained transfer rate of around 800 MB/s.
For background: http://en.wikipedia.org/wiki/SMPTE_292M
Striped raids entail the risk of data loss but increase performance.
Mirrored raids double the cost without performance benefit but with added security.
Parity raids combine the benfits, more performance and more security, but at less cost than mirrored arrays.
Would you recommend using USB 3 in place of SATA or eSATA drives?
So, to handle the uncompressed format I should set them up with one as the primary boot drive and the other three in a RAID0?
Do you know what USB stands for?
Universally Serious Backups = USB. That is all USB drives can be used for. Even with the much touted theoretical bandwidth of USB3, the USB3 disks are still around 100+ % slower than SATA and there are no raid controllers for USB drives in existence. It is all marketing hype. USB2 could not achieve more than 20-25 MB/s and it looks like USB3 will be hard pressed to achieve more than three times that performance, still lagging way behind SATA/SAS connections.
From early benchmark results I have read, the speed of USB3 drives was comparable to FW-800 drives. Things may have improved somewhat since those early tests, but still no match for SATA/eSATA.
I realize that USB 3.0 is slower than RAID, but is a 7200RPM drive, which is limited to say 125 MB/sec, going to be slower on USB 3 than SATA?
Thank you, Harm for an excellent guide!
I have the following setup:
OS: Win 7 64 - Ultimate
MB: Gigabyte GA-EX58-Extreme
CPU: Intel Core i7 975X
RAM: 24GB, (6x4 GB Mushkin DDR3 1333)
GFX: Nvidia GTX285
RAID: 3ware 9750-4i
OS Drive: 7,200 SATA 500GB
Prog Drive: 7,200 SATA 500GB
Page file and Misc.: 7,200 e-SATA 1TB
Projects, Media, Render etc.: 4 x 1TB SAS2 RAID-5
Backups: 4 x 1TB SATA RAID-5 (e-SATA)
I have one question; My read speed from the SAS raid averages about 400MB/s which I think is OK, but the write speed is about 80-100 MB/s. Is that normal, is that a result of the nature of RAID-5 over 4 disks?
That figure seems very low. With a 4 disk raid5, even on a 3Ware controller, I would have expected that to be in about or in excess of 200 MB/s.
What processes do you have running? Please post a screen shot from Process Explorer, not Task Manager. And post this in a new thread to avoid contamination.
Thanks Harm, will do once I get home!
Set your write policy for the raid volume properties to "Always Write Back"
I currently have:
Write Cache: Enabled
Read Cache: Intelligent
Auto Verify: x (checked)
Overwrite ECC: - (unchecked)
Queuing: x (checked)
I was thinking that maybe the queuing should be unchecked for SAS drives?
Are you using the Raid console management in windows or the bios raid management?
Use the one in Windows and look for a write policy setting under your raid volume properties. If you can't find it contact the manufacturer and ask them where that setting is at.
Windows 7 will continue to throttle the drive performance during write operations for data integrity until you change that setting.
For my controller (9750-4i) Write Back is called Write Cache, I believe. If I uncheck that I assume it would act as Write Through because write speed drops to 10-12 MB/s!
I am using the Web Interface through Firefox.
I would contact the manufacturer. Because the write speed your getting still sounds like the write policy is incorrect. The Cache settings are normally strictly tied to controller and drive cache and not policy settings which are separate. Disabling the cache for writing would still enable Controller cache for reading. Most controllers just give you an option to enable or disable controller cache. It looks like they split those up so you can decide which instructions get cached. That is not the same thing as write policy. It's up to you but I would contact them.
Thank you, Eric, for clarifying!
I will certainly contact 3ware/LSI and see if they can direct me in the right direction!
This is a great thread. Thanks for all the info. I am hoping that I can get some answers here. I just got my new Mac Pro last night and had it set up into a RAID 5 configuration mostly because I was not sure what I needed and the Apple guy insisted this is what I needed. I have a Mac Pro running Snow Leopard 10.6.5 three internal 1 TB hard drives with one external 2TB iOmega Hard Drive Firewire 800. I will be using this for Adobe Premiere Pro CS5 and Logic Studio.
I do primarily wedding video and some other small videos (documentaries) all on uncompressed media from Canon 5D and Canon 7D as well as video captures from other sources. I want to optimize the system. I am thinking that the RAID 5 I set up from the install disc to get all three drives in the RAID set is not the best use of this set up. I am thinking I need to put the OS and apps on one drive and then have the other two in a RAID 0 and then use my external for storage of video files and music files. But, don't know where to keep the project files and where I should do the scratch, etc. Or should I get a fourth drive, and re-do my RAID 5 to include that drive, if I do, how do I dedicate the specific drives to house the specific tasks. For example how would I get just one drive to do the OS? I understand I can dedicate the rest of the functions of the drives in the RAID set via Pr Pro after the RAID is put together.
Please provide some help if you can. It makes no sense for me to start work in this configuration since if I need to change it later, there will be much reworking of data. Figured I better get this sorted out asap so I can get up and running in the right way.
Well, you certainly have the horsepower you need, but it needs a little configuring. I shoot 5D2 footage and use Premiere Pro as well, and there are a few things to know to make this easy so it is done right the first time.
We'll discuss your Mac in a second, but first you should know that you are going to need to pull off every trick in the book to get the footage to edit smoothly. You may know a lot of this, I'm just including it so it's all in one basket (and to benefit another reader possibly). The H.264 QT format that comes out of the 5D Mark II and 7D is a good compressed format for finished video, but when we attempt to edit it, frustration can occur. You aren't editing uncompressed video by any means, if it was it would be 4K footage being that the 5D2 sensor is 5616x3744. Wouldn't that be awesome if we COULD get that out of it??
To be clearer on this, the 5D2 compresses the image down to 1920x1200 in H.264 (H.264 for video is the same thing as shooting a still image in JPEG format... we have RAW for stills but not for video YET in a 5D2). The compression used by Canon stuffs the video down into an 8-bit 4:2:0 color space which is more lossy than I would prefer, but I imagine Canon did that to make the footage as manageable as can be. In fact, with Premiere Pro CS4, it was uneditable, it just froze Premiere. To fix this, we all bought Cineform's NeoScene which converts the 5D2 footage up to a 4:2:2 10-bit file (still 8-bit but now with headroom for color grading, etc) and converts it into an .AVI file so it'll run smoothly in Windows. They do have a Mac version also, and you can buy it at VideoGuys.com for $99. ANYWAY, that's how we were able to edit in CS4. Now that CS5 is out, Adobe went to great lengths to accomodate the DSLR world by adding the right presets, etc, but most importantly by adding the MPE for all editors to leverage for all types of formats. not just DSLR vids. Now, the Mercury Playback Engine helps greatly, but as soon as you stack some effects onto the footage, it can still get jumpy. To this day I still convert everything with NeoScene because in conjuction with the MPE it's even that much better. Premiere Pro CS5 does leverage the CUDA technology in the video cards to help power that MPE, but that does mean having a compatible Nvidia graphics card, not sure if you have that in your Mac. I know there's a GTX 285 for the Mac that would work, you would have to research other models such as the uber-expensive Quadro series. I don't think there's a card in the 400 or 500 Nvidia series for Mac yet. But that may be a good reason to get NeoScene anyway.
OK, what on earth did all that have to do with your Mac's hard drives? Glad you asked! Once you ensure you have the best converted file available to use for editing (by doing all that above), you then want to place that file on your fastest drive which would be your RAID5 config. You need read speed like crazy which RAID5 will give you. You also get redundancy with that. I use RAID0 but I always back up my footage first so if RAID0 drops, I can quickly recover. I realize that doesn't help me halfway through a 6 hour encoding session, lol. ANYWAY, there's one issue with your setup: The OS and programs are also on that RAID5, correct? That's what you don't want. It'd be great if you could install a 10K RPM 300GB WD Raptor and put your OS and the Adobe Suite on that drive and save your RAID5 for your assets. I would go one step further and add one more drive to write your output video to and to also use for the Adobe Scratch Disks. Don't use partitions, and by all means don't use the external drives, FireWire is way too slow. The only external drive you would want is eSata, but really you want your footage on that RAID5 setup. I know, this all sounds kookie, you probably just wanted to get the powerhouse Mac with RAID and be done with it. That'd be fine for gaming, but for video editing you need the right setup, it will make your life much much easier.
BTW, do you have at least 12GB of RAM? We love Adobe software, but it eats ram like a cheetah eats gazelles...
OK, that's all I have. Whatever you choose to do, just make sure your footage is on that RAID5 rig, even if it shares it with the OS. Worst case, you could always buy just one additional internal drive and put all the Adobe software on it, and leave the OS on the RAID5 with your assets, because the Mac OS loads mostly into memory anyway.
Recap: Your best config if it was possible:
- 10K RPM Raptor for OS and Adobe Suite
- RAID5 for all assets
- 3rd drive for Project files and output
- 4th drive for Scratch Disks and whatever other stuff you want on your PC (Photos, etc). Keep that stuff isolated so they're not in the way of your Assets or OS. The Scratch Disk is a great place to do that.
- Use your FireWire external for backups.
This is pretty much the setup I use on the PC I built and it works well.
There it is! If I'm off track, somebody pipe in, let's get Billy up and running the right way the first time!
Message was edited by: PaulieDC... fixing typos!
Thanks for the great info!
In a three-drive set-up as described in the original post, which would drive would need to be the fastest? Which could be your slowest? (Assuming all three are 7200RPM SATA drvies of varying speeds/ages)?
Wow! Thank you very much. A wonderful New Years gift of such a thoughtful and detailed response. I feel very supported here. I think I might just be able to get this thing off the ground. Some things I need clarification on:
- I was considering getting another drive (for a fourth in the system) You are recommending a 10K RPM vs another one like the other three I have??? (Currently have three 1TB 7200 RPM SATA drives)
- Are you suggesting The OS and Adobe Pr Pro on one drive alone. What about other apps?
- When you say RAID 5 in the system are you suggesting the RAID is done on the other three discs with the OS/Pr on the other as a stand alone.
- If I have RAID set up can I still designate those drives as separate drives for Pr functions (e.g., one of the drives in the RAID is named Project and that is where the project files are written)?
I guess I failed to mention that I do have a NVIDIA Quadro 4000 for Mac. It is not installed yet, because I need to get this whole drive thing settled first before I go and install the card drivers and the CUDA drivers because it sounds like I may need to delete the RAID and start all over again with a different configuration.
Also, should mention that I have a RAID card installed in the system as well, so it will be hardware supported RAID not software.
So, how do I get started from here? Do I need to get the fourth drive, delete the current RAID 5, install the new drive, install the OS on one drive then set up RAID 5 on the remaining drives?
I would like to discuss your suggestions of formats for editing (all very good advice) but I will wait until I get this thing off the ground and then tackle that. I think once I get that all going, I will be ready to roll. Unfortunately, all of this is holding up a deadline for a publisher. But as you can see, it makes no sense to do anything until the drives are set properly so that I don't need to restart all over again since RAID set up will require a wipe and start over.
Billy, no prob, my pleasure. Just checking mail now, but will be at my PC in a little while and I'll answer your questions (and "Another Photographer's"). In the meantime, others may respond before I can... You'll find a good number of pros on here that dwarf me in knowledge, I'm more suited to answering DSLR Video questions. But either way, we'll get you going!
Awesome. I feel I am closer now. Forgot to mention, after re-reading your post, about RAM...I have 16 GB. It appears that I have all the right things (after I get fourth drive) just need to set it up correctly. Then I can get moving.
Another Photographer, your assets need to be on the fastest drive, you need lots of READ speed when editing. Your slowest can be the scratch disk drive. So in the most minimal setup, a Raptor for OS & Adobe Suite (don't need to RAID it really), then RAID0 or RAID5 for assets, and a good 7200rpm drive for scratch. If you do RAID0 for assets, you'd better have backup copies of everything on that drive stuck out on some external drive. Really, adding the third drive to make RAID5 is smartest so you get redundancy. That one extra drive is peanuts compared to what we spend on these rigs, lol!
Another Photographer, If you do RAID 5 with only 3 drives, it will distribute the OS across the set, which is my problem currently. As far as the best set up, according to Paulie and some others, is that one of the drives in the set needs to have the OS. In a three drive RAID 5 all drives are in the RAID set with the opporating system. That is how I understand it, I may be wrong as I too am still trying to figure this out. I am getting a fourth drive to do what is being suggested here, but don't know how to do that. Waiting to hear how this should be done.