Skip navigation

Storage rules for an editing rig. Some basics.

May 3, 2009 12:19 PM

  Latest reply: Gf1317, Mar 20, 2012 2:22 PM
Replies 1 2 3 4 Previous Next
  • Currently Being Moderated
    Apr 29, 2010 4:51 PM   in reply to Harm Millaard

    Harm,

     

         Thank you for your "more than timely" reply.

     

         After researching your suggestions it appears that going the way of the SuperMicro CSE-M35T drive cage and a controller card might be my best option. If you do not mind I have a few questions.

     

         If I go with an Areca ARC-1231ML card, 2 of the SuperMicro cages per machine, and 6 x 2 TB drives, will I be able to set up two independent RAID 5 arrays (3 drives each array) on the the card? This would be obviously one array for media/CS project files and the other for PF/Scaratch/Render. I have used RAID cards before but nothing in the league of the Areca is why I am asking.

     

         Assuming same drive cages, same Areca card and drives moved in to exact same positions (same SATA port as well), moving form machine A to machine B would be as easy as pulling all 6 drives and re-inserting them?

     

         Finally, if I also want external SATA access (non RAID) connectivity am I going to be forced to add another card or does Areca make a card that will fill both duties? I am looking around their site and nothing is jumping out at me. I guess I could simply add a e-SATA plate to the back of the machine and cable it to a few free SATA ports on the motherboard.

     

         And yes, the OS (Win 7 64 bit) and programs would reside on a WD Raptor.

     

         Thank you for your valuable time and insight.

     

    Kipp Litz

     
    |
    Mark as:
  • Currently Being Moderated
    May 2, 2010 5:04 PM   in reply to Harm Millaard

    So Harm, I have a 256Gb Corsair SSD (C Drive, of course), and 2 x 2Tb Seagate Sata III HDs in a RAID 0 for Media & Scatch (Preview Files in PPro). Should I be using one of the two 2Tb drives for Page & Scratch and the other strictly for media??

     
    |
    Mark as:
  • Currently Being Moderated
    May 3, 2010 3:57 AM   in reply to Harm Millaard

    Thanks Harm, again great articel.

     

    In the last mounth i followed all your articles and "rules" and have now a much faster running system.

    One thing is not clear for me

    Rule 3: 7.200 or 10.000 RPM and more single discs if possible

     

    Im using at the moment

    Drive D: pagefile, render, prev, ...

    3x WD each 300 GB 10.000 RPM raid 0 Areca controller = 900 GB 10.000 RPM raid 0

     

    to compare, would it be better to use 1 or 2 Spinpoint F1 each 1 GB 7.200 raid 0 Areca controller = 2 GB 7.200 RPM

    (BTW, variant 2 seems to be cheaper)

     

    In which relation are storage space (raid 0 configered) and RPM staying ?

     

    klfi

    from Austria   

     
    |
    Mark as:
  • Currently Being Moderated
    May 4, 2010 5:38 AM   in reply to Harm Millaard

    Thanks Harm for yr answer

     

    i think i understood 60%, my interpretation says 3x VelRap (900 GB) raid0 are  much faster than 2x Spinpoint (2 TB) raid0

     

    let me summarice:

    1. raid0 is directly porportional ... i understand

    2. redundancy.... i understand

     

    and further about acsess time:

    3. 3x VelRaptor (each 300 GB) Raid0 = 900 GB =  data transfer 300 MB/s = not understand acess time 7 msec or 3x 7 msec ??

    4. 2x Spinpoint (each 1 TB) Raid0= 2 TB = data transfer 180 MB/s = not understand acess time 13,6 msec or 2x 13,6 msec ??

     

    let me ask,

    5. whats more important, what priority we should take care of: data transfer rate or acess time

      or does the general storage rules say "as less acess time as more data transfer" - or vice versa. What are the relations?

    6. my blackmagic dicspeed tool shows me "read" and "write" transfer rate, but no acess time. what is more important for editing, read or write?  

     

    Thanks, Klaus

     
    |
    Mark as:
  • Currently Being Moderated
    May 8, 2010 10:00 AM   in reply to Harm Millaard

    When turning the indexing off do you want to do c:\ only or also the folder and subfoder option as well?

     
    |
    Mark as:
  • Currently Being Moderated
    May 8, 2010 10:21 AM   in reply to Harm Millaard

    Thanks harm...

     

    What size paging file do you recommend if you have 12gig of ram?

     
    |
    Mark as:
  • Currently Being Moderated
    May 8, 2010 11:24 AM   in reply to Harm Millaard

    Anybody care to comment on what their findings were when changing their pagefile and what size works best for 12gigs of ram.  Thanks!

     
    |
    Mark as:
  • Currently Being Moderated
    Mar 17, 2011 1:28 PM   in reply to Harm Millaard

    (Perhaps I should start a new thread)

     

    I want to optimize storage for video editing on a PC In Premiere Pro CS5, and I have not seen the information that I'm looking for after reading a lot of articles about sustained data rate being key, but, given limited budget and drives, I'm not clear where it is needed most. I am forced to reconfigure my system for unrelated reasons, so I want to get it right. My goal is to be able to avoid framedrops, hangs, or long render times. My source files are always provided on a portable firewire 7200 drive. I typically edit long files down to  20 minutes or less, with motion graphics, multiple video tracks and effects, etc.

     

    So, can someone educate me:

     

    - To minimize bottlenecks, I understand  reading and writing should be done on different disks, but I'm not clear on where the priority goes. I'm not sure what is being read and written when rendering takes place (assuming that's when it matters.) Should capture, rendering and scratching be done on 3 different disks. And if so, what's the priority for performance. When rendering is PP reading from scratch and the captured disks or just scratch at that point? Can anyone help me understand a little more here- what files are being accessed when rendering for export?

     

    - Where do I store graphics and sound effects that I want to use in the project(s)?

     

    - I have a limited number of drives - where does sustained throughput matter most? Scratch? Rendered? Source? Can you set priorities?

     

    - Also, if possible what should NOT be put on the same drive?

     

     

    Or better yet - tell me what I should do: I have a quadcore PC w 8G ram,  2x8800gtx w768M. I have 4 internal drives:  Raid0 with 2x160G drives 7200 rpm, and  I was thinking I would put my OS and apps on that. Then I have 2 internal 7200rpm 1.5G drives and I was thinking of making one for scratch and images and the other for render (both video and audio). I can use external USBs for other non-editing documents. Is this a good solution? Alternatively - if sustained throughput is more important than separating read/write, I could make a Raid0 from the 2x1.5G drives. Should I buy an esata card to connect my source media that way? (not sure I can fit the card)

     

     

    Thanks

     

    -Bill

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 5, 2011 9:12 AM   in reply to Harm Millaard

    Hi Harm,

     

    Your info is very helpful - and I understand 3 physical drives is a best case scenario. I have two questions.

     

    I am trying to run CS5.5 on a laptop with this set up:

    Dell Latitude 6520

    Intel Cor i7 2720QM CPU @ 2.20 GHz

    8.00 GB RAM (7.88 GB usable)

    Win 7 Ultimate 64-bit

     

    I currently have 2 internal drives in the machine, and both are formatted NTFS.

     

    The main drive is 450 GB and the second is 300 GB... Again realizing 3 HD is recommended, if you had to work with two - how would you align project files, raw files, media cache, and scratch disks.

     

    Also - I may be able to get an external 1TB HD that hooks up via ATA (not USB!). If I add this to the mix, how would you suggest I alter the file set up?

     

    Thanks so much!

     

    Ted

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 5, 2011 10:00 AM   in reply to Harm Millaard

    Awesome article - thanks so much!

     

    Ted

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 21, 2011 3:36 PM   in reply to Harm Millaard

    Wow, a lot of info.  I have a JVC camera that creates MTS files. I want to buy adobe premier pro and have a computer built(PC tower)  specifically for making movies. I am an amateur for sure and do this as a hobby. What kind of of optimal system requirements should I ask for to make it run fast. I hate the skipping and waiting while your machine lags behind your mind. Any advice would be welcome. Thank you 

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 22, 2011 1:49 PM   in reply to Andrew M Clark

    removed

     

    Message was edited by: Frederic Segard

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 27, 2011 7:36 PM   in reply to Harm Millaard

    Hello guys,

     

    Another FCP switcher here  ^_^

     

    I'm in the middle of building a rig, and have opted for a 6 disk areca controlled RAID3 setup similar to many threads on this forum.

     

    I am now wondering about which drives to put in there, I know the Samsungs are popular, and of course no green disks etc. 
    Would it a big enough performance increase to put 6 velociraptors in there for the price?
    Thanks for any help or advice! 
    ^_^

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 27, 2011 8:59 PM   in reply to dec0n

    Welcome fellow switcher!

     

    A good drive, tested by Areca with their controlers, very affordable, and a good all around performer, is the Hitachi 7K300 series. Raptor drives in a RAID is kind of useless. Way to expensive for very little gain (at that price, might as well take SSDs). But at roughly $65 for a 1.5TB SATA III drive, you can install drives to your heart's content (or to the maximum number of ports on your card, which ever comes first), and see a tremendous gain in speed.

     
    |
    Mark as:
  • Currently Being Moderated
    Jan 26, 2012 4:01 PM   in reply to Harm Millaard
    

     

    Rule 5: Turn off index search and compression. Both will cause severe performance hits if you leave them on.

     

     

    Sorry, I have confusion about this.  What does this refer to and where is this done?  In the OS or in Adobe?

     
    |
    Mark as:
  • Currently Being Moderated
    Jan 26, 2012 4:06 PM   in reply to RFDPiper

    RFDPiper wrote:

     

    

     

    Rule 5: Turn off index search and compression. Both will cause severe performance hits if you leave them on.

     

     

    Sorry, I have confusion about this.  What does this refer to and where is this done?  In the OS or in Adobe?

    In the OS.

     
    |
    Mark as:
  • Currently Being Moderated
    Jan 26, 2012 4:16 PM   in reply to RjL190365

    Thanks for the fast reply, but where in the OS?

     
    |
    Mark as:
  • Currently Being Moderated
    Jan 26, 2012 4:58 PM   in reply to RFDPiper

    Go to "My Computer", right-click on the disk that you want to change the settings on, select "Properties", then uncheck both of the boxes at the bottom of the "General" tab (labeled "Compress this drive to save disk space" and "Allow files on this drive to have contents indexed in addition to file properties"; by default, the latter check box is checked).

     
    |
    Mark as:
  • Currently Being Moderated
    Jan 29, 2012 7:02 PM   in reply to Harm Millaard

    Thanks RjL,

    Two more questions: 

     

    1) I have separate drives for media, OS/Programs, and render.  I am setting up my RAID 0 render drive right now (don't worry, I regularly back-up to a RAID 1 NAS), and in order for the computer to see my drive obviously I have to initialize it.  It's asking me if I want to initialize it as a MBR or a GPT.  Not sure which to choose, please help.  I'm assuming MBR is OK since I'm well under 2TB. 

     

    The drive is two 450GB Velociraptors RAIDed off of the motherboard controller if that makes a difference (I know, I know, get a hardware controller right?  Well, someday).

     

     

    2)

    Rule 1: NEVER partition a disk.

     

    I set my C: drive (OS/programs) to MBR and it seems to have partitioned itself.  Should get rid of the partition, or does that not make a difference on the C; drive?  I don't mind reinstalling my OS and everything to get the best setup

    Capture.PNG

    I'm using Windows 7 Professional 64-bit

     
    |
    Mark as:
  • Currently Being Moderated
    Jan 29, 2012 10:57 PM   in reply to RjL190365

    RjL190365 wrote:

     

    Go to "My Computer", right-click on the disk that you want to change the settings on, select "Properties", then uncheck both of the boxes at the bottom of the "General" tab (labeled "Compress this drive to save disk space" and "Allow files on this drive to have contents indexed in addition to file properties"; by default, the latter check box is checked).

    I get a promt when I uncheck the "Allow files on this drive to have contents indexed in addition to file properties" box.  Should I "Apply changes to drive C:\ only"  or  "Apply changes to drive C:\, subfolders and files"?

     
    |
    Mark as:
  • Currently Being Moderated
    Jan 30, 2012 3:41 AM   in reply to RFDPiper

    Use the later, "Apply changes to drive C:\, subfolders and files".

     

    Jim

     
    |
    Mark as:
  • Currently Being Moderated
    Feb 15, 2012 3:42 AM   in reply to Harm Millaard

    Hi Harm,

    Was wondering if you could advise. I was looking at Rule #3 on your guide. I'm not sure I can afford 3 discs so was considering a 2 Disc set up (2x 1TB Seagate Barracuda 7200rpm, 32MB Cache). I'm mainly using CS4 Ppro for HD video (<15mins in length) and some after effects. How crucial is a 3 disc set up? Will I still get good speed on that set up? And is it worth upgrading to 64MB caches?

     

    spec is:

    Intel i7 2600K 3.4GHz

    Asus P8Z68-V Pro

    8GB RAM

    Nvidia EVGA GTX 550Ti

    2x 1TB Seagate Barracuda 7200rpm, 32MB Cache

    Windows 7 Pro

     

    Thanks a lot

    Tom

     
    |
    Mark as:
  • Currently Being Moderated
    Feb 15, 2012 5:08 AM   in reply to Harm Millaard

    This is for a new system and I have limited budget so trying to work out what's best to spend my money on. How much will this affect workflow during projects or is it just the opening and saving of projects? Also, would it realistic to add a third external SATA drive at a later date if I need it?

     

    Thanks for the help,

     
    |
    Mark as:
  • Currently Being Moderated
    Feb 15, 2012 5:50 AM   in reply to Harm Millaard

    What do you reckon to an SSD for OS & Programs drive? Would you recommend? Was looking at 128GB Corsair Performance Pro

     
    |
    Mark as:
  • Currently Being Moderated
    Feb 15, 2012 7:47 AM   in reply to Harm Millaard

    yeah you have a point there. Thanks so much for the advice Harm, it's invaluable having your knowledge on tap!

     

    Okay, here's the setup i'm now looking at and the uses for each drive. Do you think this will work well?. I know it uses a green drive which you weren't a fan of but the company we're ordering through only offer those for a third drive. (unless you get an SSD for drive one) and i'm guessing this will still be superior to a 2 drive setup?

     

    Drive 1: 1TB Seagate Barracuda 7200rpm 32MB Cache.

    Drive 2: 1TB Seagate Barracuda 7200rpm 32MB Cache

    Drive 3: 1TB Western Digital Caviar Green, 64MB Cache

     

    Drive 1: OS, Programs, Pagefile

    Drive 2: Media, Projects

    Drive 3: Previews, Media Cache, Exports

     

    My other option is to go for a SSD as drive one but to keep it on budget I think that would have to mean downgrading the motherboard from a Asus P8Z68-V PRO  to a  Gigabyte GA-Z68XP-UD3 and I don't have a clue how much difference that would make to performance. Any help would be much appreciated!

     
    |
    Mark as:
  • Currently Being Moderated
    Feb 15, 2012 9:48 AM   in reply to Harm Millaard

    Okay, here's the setup i'm now looking at and the uses for each drive. Do you think this will work well?. I know it uses a green drive which you weren't a fan of but the company we're ordering through only offer those for a third drive. (unless you get an SSD for drive one) and i'm guessing this will still be superior to a 2 drive setup?

     

    Drive 1: 1TB Seagate Barracuda 7200rpm 32MB Cache.

    Drive 2: 1TB Seagate Barracuda 7200rpm 32MB Cache

    Drive 3: 1TB Western Digital Caviar Green, 64MB Cache

     

    Drive 1: OS, Programs, Pagefile

    Drive 2: Media, Projects

    Drive 3: Previews, Media Cache, Exports

     

    My other option is to go for a SSD as drive one but to keep it on budget I think that would have to mean downgrading the motherboard from a Asus P8Z68-V PRO  to a  Gigabyte GA-Z68XP-UD3 and I don't have a clue how much difference that would make to performance. Any help would be much appreciated!

     
    |
    Mark as:
  • Currently Being Moderated
    Feb 15, 2012 10:02 AM   in reply to tomglen

    In this case, get another 7200 RPM hard drive from another source: Green drives are often too slow in both sequential transfer speed and random access speed to be of much use for anything besides backups. In the case of the Caviar Green, its rotational speed is only 5400 RPM despite Western Digital's "variable-speed" claim.

     
    |
    Mark as:
  • Currently Being Moderated
    Feb 15, 2012 10:15 AM   in reply to RjL190365

    Thanks for that. Would an external eSATA connected drive be fast enough do you think?

     
    |
    Mark as:
  • Currently Being Moderated
    Feb 15, 2012 10:37 AM   in reply to tomglen

    In this case, get an external USB 3.0 or eSATA enclosure and put the Green drive into that. Then get the third internal 7200 RPM hard drive.

     
    |
    Mark as:
Actions

More Like This

  • Retrieving data ...

Bookmarked By (3)