Skip navigation
This discussion is locked
Currently Being Moderated

Generic Guideline for Disk Setup

Jun 18, 2010 5:44 AM

There have been many questions about how to set up your disks.

 

Where do I put my media cache files, where the page file, and what about my preview files?

 

All these allocations can be set in PR, so I made this overview to help you find some settings that may be beneficial. It is not a law to do it like this, it is a generic approach that would suit many users, but depending on source material, workflow and backup possibilities, it is not unthinkable you need to deviate from this approach in your individual case.

 

The reasoning behind this overview is that you want to distibute disk access across as many disks as possible and get the best performance.

 

Look for yourself:

 

Guideline Disks.jpg

 

I hope this helps to remove doubts you may have had about your setup or to find a setup that improves performance.

 
Replies 1 2 3 ... 11 Previous Next
  • Currently Being Moderated
    Jun 18, 2010 6:42 AM   in reply to Harm Millaard

    Harm,

     

    Thanks for posting this; very interesting. It's now making me question if I did the right thing in setting up my hard drives!

     

    I must admit I didn't do much research on this aspect (because I thought I was going to have burn-in time, but I had to hit the ground running with the new system!), but I think I have enough drives to squeeze out a decent bit of performance.

     

    I see that you're recommending RAID 3 or 5; I had setup my 4x 1TB Samsung F3s as a RAID 10. Did I do not such a smart thing? It looks like I could probably better utilize these for speed and storage space by perhaps changing up the way they're set up. I have been running RAID 0 with my three-year-old HP workstation for my media drives (2x 320GB) and haven't had a problem, so I really don't have any qualms about RAID 0, especially with proper backups, as you suggested. I'm using the built-in ICH10R RAID controller on the mobo for the moment, as I don't have the change for a separate hardware RAID controller right now, and I believe it only provides RAID 0, 1, 5, and 10.

     

    The OS and programs are on the VelociRaptor, and I have a 500GB/7200RPM Hitachi for projects, graphics, and the like. I usually export to the "Projects" drive, and I currently have the page file there.

     

    Any quick thoughts about how to make it better?

     

    Thanks again for the thought-provoking post...

     
    |
    Mark as:
  • Currently Being Moderated
    Jun 18, 2010 7:52 AM   in reply to Harm Millaard

    HA! Good call... yeah, I guess I was being needlessly twitchy. I've always got backups of my media before it even hits my edit drives.

     

    I was running just a two-disk RAID 0 before, which was ample for my needs at that time--of course, my system didn't know the difference, because it couldn't eat that fast. The new box has a bit more of an appetite, and everytime I walk out of the room, I swear I hear, "Feed me, Seymour!"

     

    Between the choice of a 3-disk RAID 0 + 1 extra for whatever versus a 4-disk RAID 5, you'd say take the slight performance hit, and go for peace of mind? Makes sense to me...

     
    |
    Mark as:
  • Currently Being Moderated
    Jun 18, 2010 8:03 AM   in reply to Harm Millaard

    Good tips, and an excellent rationale. I'll make it so. And I'll stop polluting your thread now

     

    Thanks, as always, Harm...

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 8, 2010 5:25 PM   in reply to Harm Millaard

    Hi Harm:

     

    Thanks for the awesome guidelines and posting on all things hardware. You 'da man.

     

    I now have a 4-disk CS5 setup (upgraded from a 2-disk CS4 setup), and will follow your guidelines. I think I understand the rationale of it all (spreading disk read/writes over multiple, low-filled physical HDDs).

     

    A few questions:

    1. Does the media cache files (assuming these to be indexes, peak files, etc.) AND the media cache database go on the same drive? (E: in my case)
    2. Does the media (video files, stills, music, etc.) AND the *.prproj go on the same drive? (D: in my case)

     

    It looks like Project Settings>Scratch Disks sets up the Captures, Video and Audio Previews in CS5. And Preferences>Media sets the "Media Cache Files" and "Media Cache Database" locations.

     

    I have a WD Raptor 10k rpm drive for my OS/Program drives and a pair of 1TB WD Black 7,200rpm drives for my E: and F:.  I have a motley mixture of SATA drives, all 7,200rpm of various sizes for my legacy projects & media.

     

    I also have a software mirror drive (a Green WD 1.5TB) that backs up necessary files every morning via MirrorFolder software. So I guess I technically have a 5-disk system.


    Thanks in advance!

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 9, 2010 7:07 AM   in reply to Harm Millaard

    Fair enough, about the disks numbers.

     

    Can you provide any help on the two questions I asked, please?

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 9, 2010 8:52 AM   in reply to Harm Millaard

    Thanks, Harm, I appreciate it!

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 9, 2010 7:54 PM   in reply to Harm Millaard

    Hi Harm,

     

    Thank you for the disk guidance.  Per your advice, I am going to reorganize my disks and get rid of my RAID 0  I have three 1TB disk, and one 320 GB disk.

     

    Which disk should I use the 320 GB one?  I'm thinking Pagefile, Media Cache.

     

    I have an ASUS P6X58D motherboard and I can use two SATA 3 drives.  Although it seems like current SATA 3 drives aren't much better than the SATA 2 drives, when I get SATA 3 drives, which ones would benefit the most for performance?

     

    Aloha,

    Roy

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 10, 2010 1:15 AM   in reply to Harm Millaard

    Hi Harm,

     

    Thanks for your advice.

     

    Aloha,

    Roy

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 12, 2010 9:41 AM   in reply to Harm Millaard

    Perfect timing for your posted guidelines, Harm.  My machine in for a rebuild after I lost TWO RAID 0 arrays at once

    (power outage while I was working, and with no UPS backup.  That won't happen again.  But I digress.

     

    I'm a little confused about how you partition the RAID arrays into logical drives in your generic setup.  If you dedicate different parts of the edit task to different physical drives when working with 4 drives or less, wouldn't you do the same with logical drives in a multi drive array?

     

    At the moment I am looking at 4 1TB drives, 2 640 GBdrives, and a 500 GB drive for a new build.  All drives are 7200 rpm drives from the same manufacturer.  I also have a bay for swapping drives for backup.

    In addition to the MoBo and a cheap Promise drive controller, I followed your suggestion and got an Areca 1210 plus the Areca battery backup.  This Areca card only supports 4 drives.  Now I have to assign the drives, and convince the guy actually doing the build to do what I want.

     

    At this point I am no longer sure that what I originally had it in mind to do is still the best thing for me. I originally thought that I would use the two 640 GB drives in a RAID 1 as the system and programs drive.  Then I was going to use one of the 1 TB drives as the drives for project files and source media, and three of the 1 TB drives in a RAID 5 for everything else.

     

    But your suggestion of only having two logical drives and maximizing the number of drives in the RAID array is forcing me to rethink that.  The Areca card I got is only good for supporting 4 drives. Should I do four drives on that card? in a RAID 3 or a RAID 5 (not sure which)?  Or use three 1TB drives in the RAID and hold the fourth one in reserve in case of failure?  Or maybe do the 4-drive array, kick out the swapping bay and add the 500 GB in for the project files?

     

    How then do I partition and assign the different editing tasks? 

    I'm muddled with the choices.

     

    Stephen

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 12, 2010 11:57 AM   in reply to Harm Millaard

    Thank you, Harm.  That arrangement would not have occurred to me, but it makes sense when I think it through.

    I will pass this on to my technician and see if I can get him to come on board with it.

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 12, 2010 11:34 PM   in reply to Harm Millaard

    harm,

    unfortunately i purchased my computer w/o fully understanding some of these hardware setup issues.  i currently have 3 hd's--

     

    750 gb = main os (wd re3)

    2, 1 tb drives = raid 0 (western digital caviar black)

     

    after looking at the guide in this thread and some of the remarks, i am feeling glum.  what would you (or anybody) suggest here?  i am not really a gearhead.  it looks like the adobe guide is suggestiing i un-raid my system and have 3 non-raid drives (or add 1 more drive).  i am not sure i can correctly do either option here, but i'd like to get my facts straight first.  help!

     

    alex

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 29, 2010 9:41 AM   in reply to polarsilver

    A big part of your decision is based on the type of files you will edit

     

    DV AVI type 2 at about 13Gig per hour or AVCHD at about 11Gig per hour do not need raid... see My CS5/AVCHD 1st Impressions http://forums.adobe.com/thread/652694?tstart=0

     

    Some other file types do need raid... but only you know what you will be editing... with AVCHD I have 3 hard drives and NO raid, and all works well

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 29, 2010 10:19 AM   in reply to John T Smith

    A raid 0 though does improve performance when exporting from either DV or AVCHD. That is something to keep in mind. You don't require it but it will help speed up your export times.

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 12:38 PM   in reply to Harm Millaard

    Appreciate the chart; most helpful but I have questions if one only has (4) drive bays total ....(yes, I'm referring to a Mac Pro!!).

     

    - Should I just follow the guide using one HDD for each function as described with no raid?

     

    - Would it be recommended to have an external setup as well?

     

    Any suggestions would be greatly appreciated.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 12:50 PM   in reply to Andrew M Clark

    Someplace there is a message thread titled (something like) "to raid, or not to raid"

     

    Using raid (mostly) depends on the type of file you will edit

     

    For AVCHD, which is "about" 11Gig per hour of video... I do not use raid

     

    For other files which are not as compressed, so the data being transferred from drive to memory is much more... a higher data rate is required

     

    For someone to try and answer your question, you need to post the file type(s) and codec(s) you will edit 

     

    ADDED

    Go here to read about RAID - http://forums.adobe.com/thread/525263

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 1:33 PM   in reply to Harm Millaard

    I shoot wedding films and I have maxed out my internal raid 5 with summer projects so I picked up a new raid encloser. Here is my new configuration that I could use some advice. I just picked up an external raid enclosure Sans Digital Tr5ut-b esata and put 4 2tb WD Blacks in there Raid 5.

     

    So now this is what I have total for Hardrives.

     

    4x  WD 2TB Black Drives (external esata raid 5)

    4x  WD 640GB Black Drives (Currently internal software Raid 5)

    1x  60Gb Vertex SSD (OS Drive)

    1x  500Gb 7200rpm Drive

     

     

    This is what I was thinking, but your advice would be much appreciated.

     

    New External Raid 5 for safe Raw Footage storage

    Converting internal raid 5 to raid 0 to use as a working disk. I was thinking of transfering project footage on there while I work on it.

    or

    3 disk raid 0 with the 4th as my drive for exports

     

    and the 500gb drive as my system backup drive.

     

    I can never figure out where to direct all the folders in premiere can you tell me where exactly to direct scratch disk folders with what ever my new set up will be?  Thanks. Where should the Captured Video and Audio Go? (I use DSLR footage so not really nessecary). Where should Video and Audio Previews go?  "Where Should Media Cache Files" and "Media Cache Database" go? Where should my pagefile go? Is there anything I am missing if so where does it go. LOL

     

    I know this has all been talked about, but as I read, there are so many configurations out there and I still keep getting confused for my configuration.

     

    Thanks.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 1:43 PM   in reply to John T Smith

    John T Smith wrote:

     

    Someplace there is a message thread titled (something like) "to raid, or not to raid"

     

    Using raid (mostly) depends on the type of file you will edit

     

    For AVCHD, which is "about" 11Gig per hour of video... I do not use raid

     

    For other files which are not as compressed, so the data being transferred from drive to memory is much more... a higher data rate is required

     

    For someone to try and answer your question, you need to post the file type(s) and codec(s) you will edit 

     

    ADDED

    Go here to read about RAID - http://forums.adobe.com/thread/525263

     

    Yes, that would be helpful wouldn't it?!

     

    Video will be primarily 8 bit source material; all flavors rangning from Hi-8 to AVCHD.

     

    Audio will be 16 bit source material

     

    Then stuff from Ae, Ai, Ps, will be utilized as well.

     

    Also if it matters, I'm using Windows 7 64bt Ultimate.

     

    Message was edited by: Shaluda

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 1:43 PM   in reply to Switch7

    Question with regards to the internal raid 5. If you lose 1 to 3 days worth of work from drive/data failure, will the cost be prohibitive to you or your company?

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 1:45 PM   in reply to ECBowen

    Not at this point no.


     

    ECBowen wrote:

     

    Question with regards to the internal raid 5. If you lose 1 to 3 days worth of work from drive/data failure, will the cost be prohibitive to you or your company?

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 1:54 PM   in reply to Switch7

    I would still not recommend a 4 drive raid 0 because of the possible failure rate, but since the cost wont be prohibitive then it would not be catastrophic if a drive failed.

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 2:02 PM   in reply to ECBowen

    I was thinking that so I could have the maximum throughput while editing. I would only make a copy of the files I am using onto the raid 0. The Project files are on the SSD Drive with a back up of those on the raid 5. So in theory if the raid 0 goes down I can still continue working from the raid 5 array. It would just be a matter of relinking to the footage on the raid 5. Correct? or am I wrong in this line of thinking.

     

    ECBowen wrote:

     

    I would still not recommend a 4 drive raid 0 because of the possible failure rate, but since the cost wont be prohibitive then it would not be catastrophic if a drive failed.

     

    Eric

    ADK

    Thanks.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 2:06 PM   in reply to Switch7

    That will minimize your loss yes as long as you keep source and exported files elsewhere as well.

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 2:10 PM   in reply to ECBowen

    Yes, but the real question is...it this the best and most efficient use of my

    ECBowen wrote:

     

    That will minimize your loss yes as long as you keep source and exported files elsewhere as well.

     

    Eric

    ADK

    drives. Or would something like a raid 10 be better? I just want to find the most effective way for speed and redundancy as possible. With all these drives.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 2:18 PM   in reply to Switch7

    Raid 5 is the best combination of speed and redundancy. I would keep the current config and not make the change to a 4 drive raid 0 unless you needed to handle uncompressed HD files or more than 5 layers R3D material at 4K.

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 2:29 PM   in reply to ECBowen

    Basically right now I run a core i7 920 OC to 4ghz, with 12ghz of

    ddr3 1600 ram and a GTX 480. I built it for CS5 and DSLR Editing, but I am still getting stutter on playback even at 1/2 resolution when I apply a simple disolve transition. It is very frustrating I am hoping the through put of raid 0 would help that. As my system often seems to get severly bogged down once the timeline starts really getting built. I have currently been trying keep my timelines minimal and tidy to work around this. But It's not always possible. I find it is the worst if I am transitioning with alot of bright or whites in the footage. Dark footage is not as bad.

     

    Basically the adobe TV MPE articles are a pipe dream to me so far.  And my windows 7 experience index rating is maxed at 7.9 across the board all except the Disk data transfer rate which is 5.9. So that seems to by my biggest bottleneck. Now whether this mean anythign to CS5 remains to be seen but If raid 0 gets me smooth playback then I am a happy lad.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 3:13 PM   in reply to Switch7

    Well your software raid could be slowing the process down so it doesn't hurt to try the raid 0. That material should be able to play 1/2 or full scale on a 920 @ 4GHz without issues. Something else is going on. BTW that HDD readout for Win 7 is just for the OS drive. Unless you have a SSD or Raptor drive it's going to read in the 5.8 or 5.9 area. Unless you have files that should not be going to your OS drive going there, then you playback should not be effected by that. I would be interested in remoting into your system and seeing how you have things setup and the Adobe configuration. Something is odd.

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 3:24 PM   in reply to ECBowen

    ECBowen wrote:

     

    Well your software raid could be slowing the process down so it doesn't hurt to try the raid 0. That material should be able to play 1/2 or full scale on a 920 @ 4GHz without issues. Something else is going on. BTW that HDD readout for Win 7 is just for the OS drive. Unless you have a SSD or Raptor drive it's going to read in the 5.8 or 5.9 area. Unless you have files that should not be going to your OS drive going there, then you playback should not be effected by that. I would be interested in remoting into your system and seeing how you have things setup and the Adobe configuration. Something is odd.

     

    Eric

    ADK

     

    Hmmm, I have a Vertex 60gb SSD as my OS and Project File Drive.

     

    I have no idea how to have you remote into my system and forgive me for declining the request as you are an internet stranger and my mom always said to not open the door to strangers.

     

    Is there anyway I can take screen shots or something for you?

     

    Thanks for the offer to help.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 3:44 PM   in reply to Switch7

    Did you have stutter when you were at full resolution?

     

    I had a problem with stutter when I first installed my EVGA GTX 275 in MPE.  It was discussed in the CS5 forum that MPE works better at full resolution.  Sure enough, I haven't had a problem with stutter since I have been using full resolution.  I am using the four HD configuration from Harm's recommendation and an i7-920.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 4:01 PM   in reply to poppabear69

    poppabear69 wrote:

     

    Did you have stutter when you were at full resolution?

     

    I had a problem with stutter when I first installed my EVGA GTX 275 in MPE.  It was discussed in the CS5 forum that MPE works better at full resolution.  Sure enough, I haven't had a problem with stutter since I have been using full resolution.  I am using the four HD configuration from Harm's recommendation and an i7-920.

     

    Yes I get the stutter at all resolutions. Unfortunately I started with the GTX 275 got frustrated so bought the Gtx 480 with minimal improvements on playback. Rendering timelines out is much faster but timeline playback stutters.

     

    I think I might go with a 4 disk raid 10 and work from there. But the question I have regarding the guidelines is: would having all working files on a raid 10 array in one place be slower than spanning them across multiple drives like the 4 drive set up? Should I keep the footage on the raid 5, run all previews and exports and media catch on the raid 10? or have another for exports as well?

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 4:03 PM   in reply to Switch7

    We have licensing with Teamviewer for remote access and that is how we remote in. You just need to download the client which is a stand alone file from www.teamviewer.com. We use Team viewer because it is easy for the client and safe. It requires a person on the other end to give us a user id and password to remote into your system. That was simply an offer because something was odd. I would definitely try what the previous poster said regarding full scale. The MPE and video card does handle scaling so it makes sense. That system you have should easily handle that material at full scale.

     

    Eric ECB

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 4:06 PM   in reply to ECBowen

    ECBowen wrote:

     

    We have licensing with Teamviewer for remote access and that is how we remote in. You just need to download the client which is a stand alone file from www.teamviewer.com. We use Team viewer because it is easy for the client and safe. It requires a person on the other end to give us a user id and password to remote into your system. That was simply an offer because something was odd. I would definitely try what the previous poster said regarding full scale. The MPE and video card does handle scaling so it makes sense. That system you have should easily handle that material at full scale.

     

    Eric ECB

     

    My SSD is the OCZ Vertex 60gb. I thought the 5.9 was odd, because when I originally ran it new it was 7.4 I think. So I did some digging and looked up Trim, I supports trim and from my understanding windows 7 automatically takes care of TRIM. Is that correct?

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 4:26 PM   in reply to Switch7

    Yes Win 7 supports Trim by default for any SSD drive that has trim support in the firmware. I was looking up that drive and reviews to see what controller is has and trim support. I could not find the controller yet but it's suppose to have trim support based on the review. Now they did not mention if they had to update the firmware for the trim support like you have to on other drives including Intel SSD. You might want to call OCZ and see if they have a firmware update for that drive. One last note was the Sequential performance results.  They were in the 160MB/s to 180MB/s range which is not bad at all. I still can't find the controller though to see if it's one that has the latency issues for asynchronous commands. I will check on OCZ's website and see if they have more info. Try downloading HDTune Pro trial and see what the SSD is reading out for Read performance currently.

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 3, 2010 4:41 PM   in reply to ECBowen

    ECBowen wrote:

     

    Yes Win 7 supports Trim by default for any SSD drive that has trim support in the firmware. I was looking up that drive and reviews to see what controller is has and trim support. I could not find the controller yet but it's suppose to have trim support based on the review. Now they did not mention if they had to update the firmware for the trim support like you have to on other drives including Intel SSD. You might want to call OCZ and see if they have a firmware update for that drive. One last note was the Sequential performance results.  They were in the 160MB/s to 180MB/s range which is not bad at all. I still can't find the controller though to see if it's one that has the latency issues for asynchronous commands. I will check on OCZ's website and see if they have more info. Try downloading HDTune Pro trial and see what the SSD is reading out for Read performance currently.

     

    Eric

    ADK

     

    It's the vertex 64gb, it does have the firmeware 1.5 that supports trim. It was new enough to come shipped with that. As for the Read performance it is average of 110 mb/s Min 71 and Max 188.

     

    But when I looked at the Health tab of HD TUNE PRO it is showing a warning in the (C7) Ultra DMA CRC Error Count. The discription says there were communication errors this may be caused by a damaged Cable.

     

    Intereting.

     
    |
    Mark as:
1 2 3 ... 11 Previous Next
Actions

More Like This

  • Retrieving data ...

Bookmarked By (14)