Generic Guideline for Disk Setup

Harm,

Thanks for posting this; very interesting. It's now making me question if I did the right thing in setting up my hard drives!

I must admit I didn't do much research on this aspect (because I thought I was going to have burn-in time, but I had to hit the ground running with the new system!), but I think I have enough drives to squeeze out a decent bit of performance.

I see that you're recommending RAID 3 or 5; I had setup my 4x 1TB Samsung F3s as a RAID 10. Did I do not such a smart thing? It looks like I could probably better utilize these for speed and storage space by perhaps changing up the way they're set up. I have been running RAID 0 with my three-year-old HP workstation for my media drives (2x 320GB) and haven't had a problem, so I really don't have any qualms about RAID 0, especially with proper backups, as you suggested. I'm using the built-in ICH10R RAID controller on the mobo for the moment, as I don't have the change for a separate hardware RAID controller right now, and I believe it only provides RAID 0, 1, 5, and 10.

The OS and programs are on the VelociRaptor, and I have a 500GB/7200RPM Hitachi for projects, graphics, and the like. I usually export to the "Projects" drive, and I currently have the page file there.

Any quick thoughts about how to make it better?

Thanks again for the thought-provoking post...

HA! Good call... yeah, I guess I was being needlessly twitchy. I've always got backups of my media before it even hits my edit drives.

I was running just a two-disk RAID 0 before, which was ample for my needs at that time--of course, my system didn't know the difference, because it couldn't eat that fast. The new box has a bit more of an appetite, and everytime I walk out of the room, I swear I hear, "Feed me, Seymour!"

Between the choice of a 3-disk RAID 0 + 1 extra for whatever versus a 4-disk RAID 5, you'd say take the slight performance hit, and go for peace of mind? Makes sense to me...

Good tips, and an excellent rationale. I'll make it so. And I'll stop polluting your thread now

Thanks, as always, Harm...

Hi Harm:

Thanks for the awesome guidelines and posting on all things hardware. You 'da man.

I now have a 4-disk CS5 setup (upgraded from a 2-disk CS4 setup), and will follow your guidelines. I think I understand the rationale of it all (spreading disk read/writes over multiple, low-filled physical HDDs).

A few questions:

1. Does the media cache files (assuming these to be indexes, peak files, etc.) AND the media cache database go on the same drive? (E: in my case)
2. Does the media (video files, stills, music, etc.) AND the *.prproj go on the same drive? (D: in my case)

It looks like Project Settings>Scratch Disks sets up the Captures, Video and Audio Previews in CS5. And Preferences>Media sets the "Media Cache Files" and "Media Cache Database" locations.

I have a WD Raptor 10k rpm drive for my OS/Program drives and a pair of 1TB WD Black 7,200rpm drives for my E: and F:.  I have a motley mixture of SATA drives, all 7,200rpm of various sizes for my legacy projects & media.

I also have a software mirror drive (a Green WD 1.5TB) that backs up necessary files every morning via MirrorFolder software. So I guess I technically have a 5-disk system.

Thanks in advance!

Fair enough, about the disks numbers.

Can you provide any help on the two questions I asked, please?

Thanks, Harm, I appreciate it!

Hi Harm,

Thank you for the disk guidance.  Per your advice, I am going to reorganize my disks and get rid of my RAID 0  I have three 1TB disk, and one 320 GB disk.

Which disk should I use the 320 GB one?  I'm thinking Pagefile, Media Cache.

I have an ASUS P6X58D motherboard and I can use two SATA 3 drives.  Although it seems like current SATA 3 drives aren't much better than the SATA 2 drives, when I get SATA 3 drives, which ones would benefit the most for performance?

Aloha,

Roy

Hi Harm,

Thanks for your advice.

Aloha,

Roy

Perfect timing for your posted guidelines, Harm.  My machine in for a rebuild after I lost TWO RAID 0 arrays at once

(power outage while I was working, and with no UPS backup.  That won't happen again.  But I digress.

I'm a little confused about how you partition the RAID arrays into logical drives in your generic setup.  If you dedicate different parts of the edit task to different physical drives when working with 4 drives or less, wouldn't you do the same with logical drives in a multi drive array?

At the moment I am looking at 4 1TB drives, 2 640 GBdrives, and a 500 GB drive for a new build.  All drives are 7200 rpm drives from the same manufacturer.  I also have a bay for swapping drives for backup.

In addition to the MoBo and a cheap Promise drive controller, I followed your suggestion and got an Areca 1210 plus the Areca battery backup.  This Areca card only supports 4 drives.  Now I have to assign the drives, and convince the guy actually doing the build to do what I want.

At this point I am no longer sure that what I originally had it in mind to do is still the best thing for me. I originally thought that I would use the two 640 GB drives in a RAID 1 as the system and programs drive.  Then I was going to use one of the 1 TB drives as the drives for project files and source media, and three of the 1 TB drives in a RAID 5 for everything else.

But your suggestion of only having two logical drives and maximizing the number of drives in the RAID array is forcing me to rethink that.  The Areca card I got is only good for supporting 4 drives. Should I do four drives on that card? in a RAID 3 or a RAID 5 (not sure which)?  Or use three 1TB drives in the RAID and hold the fourth one in reserve in case of failure?  Or maybe do the 4-drive array, kick out the swapping bay and add the 500 GB in for the project files?

How then do I partition and assign the different editing tasks? 

I'm muddled with the choices.

Stephen

Thank you, Harm.  That arrangement would not have occurred to me, but it makes sense when I think it through.

I will pass this on to my technician and see if I can get him to come on board with it.

harm,

unfortunately i purchased my computer w/o fully understanding some of these hardware setup issues.  i currently have 3 hd's--

750 gb = main os (wd re3)

2, 1 tb drives = raid 0 (western digital caviar black)

after looking at the guide in this thread and some of the remarks, i am feeling glum.  what would you (or anybody) suggest here?  i am not really a gearhead.  it looks like the adobe guide is suggestiing i un-raid my system and have 3 non-raid drives (or add 1 more drive).  i am not sure i can correctly do either option here, but i'd like to get my facts straight first.  help!

alex

A big part of your decision is based on the type of files you will edit

DV AVI type 2 at about 13Gig per hour or AVCHD at about 11Gig per hour do not need raid... see My CS5/AVCHD 1st Impressions http://forums.adobe.com/thread/652694?tstart=0

Some other file types do need raid... but only you know what you will be editing... with AVCHD I have 3 hard drives and NO raid, and all works well

A raid 0 though does improve performance when exporting from either DV or AVCHD. That is something to keep in mind. You don't require it but it will help speed up your export times.

Eric

ADK

Appreciate the chart; most helpful but I have questions if one only has (4) drive bays total ....(yes, I'm referring to a Mac Pro!!).

- Should I just follow the guide using one HDD for each function as described with no raid?

- Would it be recommended to have an external setup as well?

Any suggestions would be greatly appreciated.

Someplace there is a message thread titled (something like) "to raid, or not to raid"

Using raid (mostly) depends on the type of file you will edit

For AVCHD, which is "about" 11Gig per hour of video... I do not use raid

For other files which are not as compressed, so the data being transferred from drive to memory is much more... a higher data rate is required

For someone to try and answer your question, you need to post the file type(s) and codec(s) you will edit 

ADDED

Go here to read about RAID - http://forums.adobe.com/thread/525263

I shoot wedding films and I have maxed out my internal raid 5 with summer projects so I picked up a new raid encloser. Here is my new configuration that I could use some advice. I just picked up an external raid enclosure Sans Digital Tr5ut-b esata and put 4 2tb WD Blacks in there Raid 5.

So now this is what I have total for Hardrives.

4x  WD 2TB Black Drives (external esata raid 5)

4x  WD 640GB Black Drives (Currently internal software Raid 5)

1x  60Gb Vertex SSD (OS Drive)

1x  500Gb 7200rpm Drive

This is what I was thinking, but your advice would be much appreciated.

New External Raid 5 for safe Raw Footage storage

Converting internal raid 5 to raid 0 to use as a working disk. I was thinking of transfering project footage on there while I work on it.

or

3 disk raid 0 with the 4th as my drive for exports

and the 500gb drive as my system backup drive.

I can never figure out where to direct all the folders in premiere can you tell me where exactly to direct scratch disk folders with what ever my new set up will be?  Thanks. Where should the Captured Video and Audio Go? (I use DSLR footage so not really nessecary). Where should Video and Audio Previews go?  "Where Should Media Cache Files" and "Media Cache Database" go? Where should my pagefile go? Is there anything I am missing if so where does it go. LOL

I know this has all been talked about, but as I read, there are so many configurations out there and I still keep getting confused for my configuration.

Thanks.

John T Smith wrote:

 

Someplace there is a message thread titled (something like) "to raid, or not to raid"

 

Using raid (mostly) depends on the type of file you will edit

 

For AVCHD, which is "about" 11Gig per hour of video... I do not use raid

 

For other files which are not as compressed, so the data being transferred from drive to memory is much more... a higher data rate is required

 

For someone to try and answer your question, you need to post the file type(s) and codec(s) you will edit 

 

ADDED

Go here to read about RAID - http://forums.adobe.com/thread/525263

Yes, that would be helpful wouldn't it?!

Video will be primarily 8 bit source material; all flavors rangning from Hi-8 to AVCHD.

Audio will be 16 bit source material

Then stuff from Ae, Ai, Ps, will be utilized as well.

Also if it matters, I'm using Windows 7 64bt Ultimate.

Message was edited by: Shaluda

Question with regards to the internal raid 5. If you lose 1 to 3 days worth of work from drive/data failure, will the cost be prohibitive to you or your company?

Eric

ADK

Not at this point no.

ECBowen wrote:

 

Question with regards to the internal raid 5. If you lose 1 to 3 days worth of work from drive/data failure, will the cost be prohibitive to you or your company?

 

Eric

ADK

I would still not recommend a 4 drive raid 0 because of the possible failure rate, but since the cost wont be prohibitive then it would not be catastrophic if a drive failed.

Eric

ADK

I was thinking that so I could have the maximum throughput while editing. I would only make a copy of the files I am using onto the raid 0. The Project files are on the SSD Drive with a back up of those on the raid 5. So in theory if the raid 0 goes down I can still continue working from the raid 5 array. It would just be a matter of relinking to the footage on the raid 5. Correct? or am I wrong in this line of thinking.

ECBowen wrote:

 

I would still not recommend a 4 drive raid 0 because of the possible failure rate, but since the cost wont be prohibitive then it would not be catastrophic if a drive failed.

 

Eric

ADK

Thanks.

That will minimize your loss yes as long as you keep source and exported files elsewhere as well.

Eric

ADK

Yes, but the real question is...it this the best and most efficient use of my

ECBowen wrote:

 

That will minimize your loss yes as long as you keep source and exported files elsewhere as well.

 

Eric

ADK

drives. Or would something like a raid 10 be better? I just want to find the most effective way for speed and redundancy as possible. With all these drives.

Raid 5 is the best combination of speed and redundancy. I would keep the current config and not make the change to a 4 drive raid 0 unless you needed to handle uncompressed HD files or more than 5 layers R3D material at 4K.

Eric

ADK

Basically right now I run a core i7 920 OC to 4ghz, with 12ghz of

ddr3 1600 ram and a GTX 480. I built it for CS5 and DSLR Editing, but I am still getting stutter on playback even at 1/2 resolution when I apply a simple disolve transition. It is very frustrating I am hoping the through put of raid 0 would help that. As my system often seems to get severly bogged down once the timeline starts really getting built. I have currently been trying keep my timelines minimal and tidy to work around this. But It's not always possible. I find it is the worst if I am transitioning with alot of bright or whites in the footage. Dark footage is not as bad.

Basically the adobe TV MPE articles are a pipe dream to me so far.  And my windows 7 experience index rating is maxed at 7.9 across the board all except the Disk data transfer rate which is 5.9. So that seems to by my biggest bottleneck. Now whether this mean anythign to CS5 remains to be seen but If raid 0 gets me smooth playback then I am a happy lad.

Well your software raid could be slowing the process down so it doesn't hurt to try the raid 0. That material should be able to play 1/2 or full scale on a 920 @ 4GHz without issues. Something else is going on. BTW that HDD readout for Win 7 is just for the OS drive. Unless you have a SSD or Raptor drive it's going to read in the 5.8 or 5.9 area. Unless you have files that should not be going to your OS drive going there, then you playback should not be effected by that. I would be interested in remoting into your system and seeing how you have things setup and the Adobe configuration. Something is odd.

Eric

ADK

ECBowen wrote:

 

Well your software raid could be slowing the process down so it doesn't hurt to try the raid 0. That material should be able to play 1/2 or full scale on a 920 @ 4GHz without issues. Something else is going on. BTW that HDD readout for Win 7 is just for the OS drive. Unless you have a SSD or Raptor drive it's going to read in the 5.8 or 5.9 area. Unless you have files that should not be going to your OS drive going there, then you playback should not be effected by that. I would be interested in remoting into your system and seeing how you have things setup and the Adobe configuration. Something is odd.

 

Eric

ADK

Hmmm, I have a Vertex 60gb SSD as my OS and Project File Drive.

I have no idea how to have you remote into my system and forgive me for declining the request as you are an internet stranger and my mom always said to not open the door to strangers.

Is there anyway I can take screen shots or something for you?

Thanks for the offer to help.

Did you have stutter when you were at full resolution?

I had a problem with stutter when I first installed my EVGA GTX 275 in MPE.  It was discussed in the CS5 forum that MPE works better at full resolution.  Sure enough, I haven't had a problem with stutter since I have been using full resolution.  I am using the four HD configuration from Harm's recommendation and an i7-920.

poppabear69 wrote:

 

Did you have stutter when you were at full resolution?

 

I had a problem with stutter when I first installed my EVGA GTX 275 in MPE.  It was discussed in the CS5 forum that MPE works better at full resolution.  Sure enough, I haven't had a problem with stutter since I have been using full resolution.  I am using the four HD configuration from Harm's recommendation and an i7-920.

Yes I get the stutter at all resolutions. Unfortunately I started with the GTX 275 got frustrated so bought the Gtx 480 with minimal improvements on playback. Rendering timelines out is much faster but timeline playback stutters.

I think I might go with a 4 disk raid 10 and work from there. But the question I have regarding the guidelines is: would having all working files on a raid 10 array in one place be slower than spanning them across multiple drives like the 4 drive set up? Should I keep the footage on the raid 5, run all previews and exports and media catch on the raid 10? or have another for exports as well?

We have licensing with Teamviewer for remote access and that is how we remote in. You just need to download the client which is a stand alone file from www.teamviewer.com. We use Team viewer because it is easy for the client and safe. It requires a person on the other end to give us a user id and password to remote into your system. That was simply an offer because something was odd. I would definitely try what the previous poster said regarding full scale. The MPE and video card does handle scaling so it makes sense. That system you have should easily handle that material at full scale.

Eric ECB

ECBowen wrote:

 

We have licensing with Teamviewer for remote access and that is how we remote in. You just need to download the client which is a stand alone file from www.teamviewer.com. We use Team viewer because it is easy for the client and safe. It requires a person on the other end to give us a user id and password to remote into your system. That was simply an offer because something was odd. I would definitely try what the previous poster said regarding full scale. The MPE and video card does handle scaling so it makes sense. That system you have should easily handle that material at full scale.

 

Eric ECB

My SSD is the OCZ Vertex 60gb. I thought the 5.9 was odd, because when I originally ran it new it was 7.4 I think. So I did some digging and looked up Trim, I supports trim and from my understanding windows 7 automatically takes care of TRIM. Is that correct?

Yes Win 7 supports Trim by default for any SSD drive that has trim support in the firmware. I was looking up that drive and reviews to see what controller is has and trim support. I could not find the controller yet but it's suppose to have trim support based on the review. Now they did not mention if they had to update the firmware for the trim support like you have to on other drives including Intel SSD. You might want to call OCZ and see if they have a firmware update for that drive. One last note was the Sequential performance results.  They were in the 160MB/s to 180MB/s range which is not bad at all. I still can't find the controller though to see if it's one that has the latency issues for asynchronous commands. I will check on OCZ's website and see if they have more info. Try downloading HDTune Pro trial and see what the SSD is reading out for Read performance currently.

Eric

ADK

ECBowen wrote:

 

Yes Win 7 supports Trim by default for any SSD drive that has trim support in the firmware. I was looking up that drive and reviews to see what controller is has and trim support. I could not find the controller yet but it's suppose to have trim support based on the review. Now they did not mention if they had to update the firmware for the trim support like you have to on other drives including Intel SSD. You might want to call OCZ and see if they have a firmware update for that drive. One last note was the Sequential performance results.  They were in the 160MB/s to 180MB/s range which is not bad at all. I still can't find the controller though to see if it's one that has the latency issues for asynchronous commands. I will check on OCZ's website and see if they have more info. Try downloading HDTune Pro trial and see what the SSD is reading out for Read performance currently.

 

Eric

ADK

It's the vertex 64gb, it does have the firmeware 1.5 that supports trim. It was new enough to come shipped with that. As for the Read performance it is average of 110 mb/s Min 71 and Max 188.

But when I looked at the Health tab of HD TUNE PRO it is showing a warning in the (C7) Ultra DMA CRC Error Count. The discription says there were communication errors this may be caused by a damaged Cable.

Intereting.