Skip navigation

To RAID or not to RAID, that is the question

Nov 16, 2009 3:46 PM

  Latest reply: Lyndon1984, Jul 23, 2012 3:18 PM
Replies 1 2 3 Previous Next
  • Currently Being Moderated
    Mar 18, 2011 2:02 AM   in reply to dreinz

    dreinz wrote:

     

    Can someone please help me out with my RAID/BACKUP solution and answer the above message posting. PLEASE! AND THANK YOU!

    You might as well start a new thread, coz this one seems old!

     
    |
    Mark as:
  • Currently Being Moderated
    Mar 18, 2011 2:34 AM   in reply to dreinz

    dreinz wrote:

     

    Can someone please help me out with my RAID/BACKUP solution and answer the above message posting. PLEASE! AND THANK YOU!

     

    Check out these links by Harm.

     

    1. http://forums.adobe.com/thread/427772
    2. http://forums.adobe.com/thread/662972

     

    They might help in your case. Harm is a qualified advisor on PCs and not Macs. Check out this link http://forums.adobe.com/thread/598040

     
    |
    Mark as:
  • Currently Being Moderated
    May 9, 2011 1:57 AM   in reply to Harm Millaard

    This was written a while back, and I am wondering if it is a viable question still; in other words - my question is: is RAIDing still considered necessary for video editing? And if so, can someone suggest a Raid Controller?

     

    This is intended for a machine on which i'd edit video that would go on the web, (hence NOT HD), though it needs to be decent quality. And, the machine would need to be good for pixel and vector graphics creation, as well as Flash, which, i believe are less demanding.

     

    thanks.

     
    |
    Mark as:
  • Currently Being Moderated
    May 9, 2011 7:53 AM   in reply to minimalcomfort

    raid or not is determined by your codecs and redundacy needs

     

    EG: DV/HDV does not need raid for performance, nor would it benefit if you did raid but you may want it for redundacy (nothing beats an external back up regardless)

     

    AVCHD: can squeak by without it but vastly better with. however going from 2 x 2 drive raid 0 to 2 x 4 drive raid 0 gains you nothing and is a waste (or 8 drive raid 3,5,6) with the later offering rundancy at least.

     

    Red 4K  and uncomcompressed has to have raid and a big one.

     

    so cant answer your question without knowing what codec

     

    Scott

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    May 10, 2011 2:40 AM   in reply to Scott Chichelli

    hi Scott. Thanks for your reply.

     

    Maybe i should say a bit about myself in order to remove any expectation that i should in fact comprehend that reply, or to  save face (my own, of course), regarding my lack in hardware savvyness.

     

    That is, i am struggling to understand your comments because i am quite devoid of technical/hardware experience and knowlege, possibly because of the type of video work that i've been doing, possibley because of Apple (: -  Macintosh doesn't tender much in terms of enabling one to specialize/or setup a machine; for a long while i was using Final Cut Pro/Studio, on a variety of Macs, shooting on a Sony Semi-Professional camera (bought in the year 2000,) and using a DSR-11 tape for digitizing the footage, to make 2-9 minute video art clips.

     

    After that, i downgraded the quality of my output (because my DSR-11 broke down), so i was using an old digital Olypus camera (c-5050) which produces QuickTime files, editing those in Final Cut to make my little movies, which i would convert to flv (Flash video), to incorporate in interactive swf files (small-web-file that's exported via Flash -  for anyone who doesn't know).

     

    At this time, i seek to build a robust machine, to handle commercial quality video output (ideally), along with my programming (Flash ActionScript) and designing aspirations (Illustrator graphics).

     

    So, going over your reply:

    if I understand correctly, you suggest that RAIDing benefits redundancy. Not sure what you mean by redundancy here, is it that during compression and export to movies, after editing a sequence, the files which are used in the edited sequence will maintain their quality - detail due to redundancy? And how does that relate to backing up data.

     

    Please excuse my ineptitude - am quite aware of it here, my knowlege about RAIDing is new and my source is YouTube such as - http://www.youtube.com/watch?v=zpvUQUIzNDA&feature=related

     

    Following  Generic Guideline for Disk Setup  (http://forums.adobe.com/message/3663705#3663705)

    it seems that RAID5 is ultimately better than RAID0, a discussion, by the way, that certainly pertains to the question of RAIDing at all, but seems to present so many threads of info and dilemma among the participants, that i keep losing focus, though it seems an productive source for me, in finalizing my decision about RAIDing or NOT.

     

    Re: your comment on "Red 4K  and uncomcompressed" - again, sorry, no idea what that is, is that a codec?

    And as for which codec i'm using, i hope by now you "see" that i'm unable to answer that. (I do NOT have a proper editing machine as yet.)

     

    I would greatly appreciate your patience, as that of any other participant, in helping me educate sufficiently in order to decide on the components for the    new machine which i intend to build.

     

    Thanks again,

    best

     
    |
    Mark as:
  • Currently Being Moderated
    May 10, 2011 6:14 AM   in reply to minimalcomfort

    HI,

     

    "codec" is determined by what cam you are using. (also there is ways to upscale etc but that’s another discussion)

     

    Are you still using the old Olympus? It uses motion jpeg and very small files at that.

     

    No need for raid.

     

     

     

    Redundancy: in a raid 3,5,6  a hard drive can die and you do not lose your files

     

    In raid 0 you can lose your work

     

    In raid 10 you are backed up as well but I don’t trust or like that raid.

     

    Regardless of what raid you have its very prudent to have an external back up. Even with raid 6 (you can lose 2 hdds and not lose data)

     

     

     

    Raid 0,10, 3, 5 ,6 (and others like raid 50) all increase speed where the more drives you have the better the speed gets.

     

     

     

    But again if your codec/workflow does not need the speed you spend money for nothing..

     

     

     

    This is probably the biggest "MYTH" perpetrated in this and other forums. The need for absurd raid 3,5,6 arrays.

     

    For the average guy the only benefit is redundancy (back up) and at what price? A cheap external backup is far less costly.

     

     

     

    Unless you buy a new cam you don’t need raid and if you do buy, it will most likely be an AVC variant that only needs 2 x 2 raid 0 (and ext back up)

     

    (something you can add later) and no need for an expensive raid card either.

     

     

     

    Hope that helps

     

     

     

    Scott

     

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    May 11, 2011 12:25 AM   in reply to Scott Chichelli

    Hi Scott.

     

    So, from you i understand that the codec type determines if to RAID or NOT. And from Wikipedia, that codecs are devices or programs used for digitizing - converting/encoding media from one to another format, like when using a digital video cassette recording machine such as the Sony DSR-11, to get the footage on a tape (shot in a video camcorder,) into a computer in order to edit that footage.

     

    With my Olympus there was no need for digitizing, as you said, the files are quite small, u simply drop them into Final Cut. Hence no Raiding?

     

    In fact, i recently got a pocket sized camera - Sony Cyber-Shot, (lovely little thing) but have not edited footage shot with it. Can you help me find out which codec i'll be needing for that? I am using this camera a lot and no doubt will do so in future.

     

     

    Nevertheless, if i did have a capable machine for editing video, i would likely go back to using my Sony TRV900 camcorder which demands digitizing (encoding) the media. Alternatively, i'd borrow a similar but more professional camcorder, or buy one sometime in the next 2 years. SO, i am considering this in deliberating the capacity of the new machine, RAIDing or not...

     

    In any case, I didnt understan what you said AVC being a variant that only needs 2 x 2 raid 0 (and ext back up). And my concerns are deepening Re the way you describe 2 x 2 raid0. No idea what that means. Is that to do with disc setup, like having 4 relatively cheap hard discs, a Solid State Drive for software, and a RAID controller? which brings me back to my original question about which RAID controllers to look at.

     

    Re the question of RAIDing in terms of backup or redundancy - i cant think beyond the term "redundancy" which you are  using synonymously with backup, and if u can refer me to some sources to learn about that i'd really appreciate it... (i do always backup media, but never used EDL lists - that's in Final Cut, not sure what the Premier equivalent)....

     

    thanks for your patience,

    best

    MC

     
    |
    Mark as:
  • Currently Being Moderated
    May 11, 2011 6:58 AM   in reply to minimalcomfort

    Hi,

    Your old cam is DV only and does not require raid, no cam you have requires raid at all.

     

    Again if you buy a new cam that most likely will require raid. (you can add the 2 extra drives later)

     

    If you back up your work you don’t require raid 3,5,6. Ideally you back up to a NAS that is raid 5 but that’s a luxury not all have.

     

    Yes redundancy means back-up in a sense or security in that a drive can die without losing data again anyone with smarts still backs up their raid to external.

     

    For now you do not need raid and I don’t see you requiring any large raids later. (in the near future)

    So you don’t need a raid controller period.

    If you end up needing the 2 sets raid 0 the onboard controller is more than fine.

     

    SSD is complete waste of money for an OS or anything else unless doing animation.

    Flash would not be considered animation.

     

    So

    OS/programs  drive

    Media drive

    Export drive

    External back up

     

    That’s it don’t complicate it or waste money.

    Put that money toward a decent cam....

     

    Scott

     
    |
    Mark as:
  • Currently Being Moderated
    May 11, 2011 8:59 PM   in reply to minimalcomfort

    RAID = Redundant Array of Independent Disks.  Check out Wikipedia's descriptions - they are not comprehensive, but decent to get a sense of the redundancy (and therefore data security) and/or access speed aspects that different types of a RAID configurations offer.  Keep in mind that the array of disks appears as a "single disk" to the operating system, so whereas there may be two or more independent disks in the array, it will "appear" to be a single disk (e.g., "D" or "E" etc.)  In some types of RAID configurations, the data redundancy (security) is within the array itself so that if one of the component disks crashes, the stored data is safe and can be accessed.

     

    On the other hand, a RAID offers no real "backup" protection - if your power supply throws a spark and your computer, case and RAID drives all burn up, any redundancy in the RAID won't be of any real use; however, a backup image of the disk(s) that is stored offsite would be.  That's why redundancy and backup strategies are treated separately.

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 29, 2011 1:38 AM   in reply to Harm Millaard

    Greetings, Harm ~

     

    I'm building a new external RAID with an Areca 1880ix-12 and a Sans Digital TR8X.

     

    People keep telling me RAID 3 is obsolete, and I should use RAID 5. I feel like RAID 3 makes more sense, so I'm sticking with it.

     

    My questions are:

     

    1. If I'm using six 2TB WD2003FYYS (RE4) discs, does it matter if I choose RAID 3 or RAID 30? I see a lot have chosen 6x2TB RAID 3, and I wonder why they don't set up 3x2TB RAID 3 + 3x2TB RAID 3 in a RAID 0 = RAID 30. I'm guessing RAID 30 is better for more than six discs, since RAID 3 requires a minimum of three discs.

     

    2. Does having an extra disc in the tower standing by as a hot-swap in case of failure shorten the life of that disc significantly, given it's only spinning and not working? Having eight bays in the TR8X, I will have eight of those WD2003FYYS discs on hand at all times in case of a drive failure, leaving one of the two extra drives in the tower as a hot-swap and the other in its box to minimize wear, and I'm wondering if I should just leave both extras in, or maybe even use 7x2TB RAID 3 instead.

     

    Thank you!

     

    ~ Marcus

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 29, 2011 8:30 AM   in reply to wonderspark

    Marcus,

     

    I've always been a bit curious about the Sans Digital TR8X unit that you are going with, and I have a few questions - two of which you may not be able to answer just yet:

    1) What connector / method are you using to convert one of your internal SFF-8087 connectors to work with the TR8X's SFF-8088 cables?

    2) How quiet / noisy is it?

    3) What are the max. temps. tlhat your 2TB's see when working hard; say, after a 1 hour render?

     

    I have a few suggestions to pass along too that has been mentioned before on this forum, I think by Bill Gehrke for setting up a new RAID:

    - run a full and write pass on each drive individually to make sure each is working properly; I like HD Tune Pro with these settings: (Partial test, accurate, 2MB block size). This will flush out any weak drive BEFORE the time and effort has be done to build and load the full RAID array

    - configure your planned RAID as RAID 0 and put it to work with some video files; I like to use PPBM5 at this stage. Why do this? Because it only takes a few minutes to do (RAID 0 builds instantly) and you will be able to test all aspects of your build (connectors, power supply, drive spin up times, drive sleeping if you will be doing that - I do on my Areca w/ WD RE3, WD Blacks, Hitachi 3TB 7k3000's, and Hitachi 1TB 7k1000.C's now, controller card under stress, etc.)

     

    Thanks in advance,

     

    Jim

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 29, 2011 8:43 AM   in reply to Harm Millaard

    Hey Harm,

    ever wonder wny NONE of the video raid drive manufacturers offer raid 3?

     

     

    rebuild times? ok

     

    http://forums.adobe.com/message/3461646#3461646

     

    Scott

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 29, 2011 9:10 AM   in reply to JEShort01

    JEShort01 wrote::

    1) What connector / method are you using to convert one of your internal SFF-8087 connectors to work with the TR8X's SFF-8088 cables?

    Thanks in advance,

     

    Jim

    Jim is this what you are looking for?

     

    If not these people at PCPitStop can problably come up with it   And the ARC-1880ix-12 has an external SFF-8088 connector

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 29, 2011 10:04 AM   in reply to Bill Gehrke

    Bill,

     

    I'm aware that the IX-12 has one 8088 external connector, but Marcus will need two for his Sans Digital tower. That cable you point to (expensive cable too BTW!) would indeed do the job, thanks for the link. It would need to connect to an internal port on the Areca and then pass through the PC case somewhere to get to the Sans Digital device.

     

    When I was thinking about doing something like this a year ago, I figured that a bracket like this (yes, I've ordered from PC PitStop before) could work too:

    http://www.pc-pitstop.com/sas_cables_adapters/AD8788-1.asp

     

    This way, all cables could be disconnected (cleaner), and the two SFF-8088 cables (included with the Sans Digital unit) connecting the PC to the Sans Digital tower would be identical; I was however concerned about having all those connections for RAID that you want to be reliable (from internal Areca connection: 8087 to 8087 cable, 8087/8088 bracket, 8088/8088 external cable, 8088/??? connection on the back of the Sans Digital, Sans Digital backplane, connect to the drives).

     

    As with many choices we have to make there are compromises!

     

    So, still curious, what are you planning Marcus?

     

    Jim

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 29, 2011 11:15 AM   in reply to JEShort01

    Jim ~

     

    My plan was to use the internal->external cable from PC-PitStop, but now that you've shown me that bracket adapter, I'll have to consider that as well. It would be nice to have a cleaner look. I was a bit concerned with how using one internal and one external cable might perform together, but my thought was that if there was an issue with it, I'd just use two internal ->external cables instead of using the external at all. I had considered using the 1880x card with two external ports, but chose the ix12 since it seems more capable of expansion, should I need that. (I hope not! This one is costing me enough as it is.)

     

    Harm ~

     

    Yes, you're right... I meant hot-spare, not hot-swap. I was sleepy. (:

    Thank you for clearing up the RAID3 vs RAID30 question. I tried to do all my homework and research all this, yet I still get confused along the way. I read that a RAID30 is faster due to the stripe, but you indicate that the RAID3 has faster throughput.

     

    After some thought, I think I understand now - - a RAID3 will use only one parity disk whereas a RAID30 will have two parity disks, thus the decreased capacity and throughput on the 30. For some reason, I figured for every three disks in a RAID3, a parity disk would be built. Surely you couldn't have, say, ten disks in RAID3 with nine being used for data and only one for parity. At some point more parity is needed, and somewhere there is an optimal number of disks to use in RAID3. In that way, I can see using more than six disks in a RAID30 to better utilize that parity issue. Do I have it right?

     

    Perhaps it's this parity problem that perpetuates the idea that RAID3 is obsolete. My confusion tempts me to avoid it and just go with RAID5, but your performance numbers seem solid, so I have to at least give it a try.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 1, 2011 12:11 AM   in reply to JEShort01

    Jim ~

     

    I've been warned against using adapters, so I'm just going to use the 1m SFF-8087 to SFF-8088 cable through an uncovered PCI slot.

     

    Can't wait to get all my parts in and test it out!

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 7, 2011 7:51 AM   in reply to Harm Millaard

    I am not happy with the proformance of my RAID system. What do y'all know about the Raid Rocket 4320 Hardware PCIe card? I am usinmg (4) 74Gb SAS 10k in RAID3 (I think that's correct). The RAID3 is for my D drive and I boot C off of another larger SAS 10k drive. Too often, it appears like my disk system goes off to think for a few seconds before anything changes on the display!

     

    Has it gotten to the point that RAID is not required as much due to the larger & faster drives coming out these days?

     

    Suggestions please.

     

    Thanks,

    William

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 7, 2011 9:41 AM   in reply to Harm Millaard

    Hi Harm, your articles have been informative. Could you offer some suggestions configuring a RAID with the following:

     

    Asus Rampage III Formula:  2 Sata 3 ports and 6 Sata 2 ports. Would you put the OS on one of the Sata 2 or Sata 3?

     

    http://www.asus.com/Motherboards/Intel_Socket_1366/Rampage_III_Formula /#specifications

     

    I have 6 Hatachi 7k3000. (plan to add more and work around these)

     

    There is room for 9 HDD in the case.

     

    I've noticed in your set up you have 2 drives in Raid  0 off the MB, and a Raid card for another raid.

     

    I probably want to add a Raid card when I do this. So would it best to have the OS on a Intel Sata II plug, (1 of 6) two Hatachi in Raid 0 on Sata 6 (if it can do raid?) and the remaining 5 Sata II ports for Raid 3 with raid card.

     

    I suppose another option is OS on Sata III port and configure the remaining 5 or 6 Sata II ports for Raid 3. That would leave one Sata III port.

     

    That's the limit of my uderstanding. I havn't configured any Raid so don't know if I'm underthinking this or over thinking it.

     

    So far my experience in Bios the 6 Sata II ports have IDE, ACHI, or Raid. But I've read that it can be set up for both ACHI and Raid. I'm not sure how to set this up. Or the Sata III ports. For example can a Raid 3 be on both the Sata II and Sata III ports? I've read that only Sata II ports (Intel) are Raid.

     

    Jim also suggested a Intel utility for switching IDE, ACHI, Raid but I couldn't find it.

     

    Also not sure what the limitations are, for example can you have C: on Sata III, 4 Sata II ports for Raid 3 with Raid card, and 2 Sata ports for Raid 0 on MB?

     

    Anyway, I'm trying to figure out how the Sata II ports and Sata III ports are compatible with Raid on MB and Raid Card for editing with PP5.5.

     

    Thanks if anyone can offer any suggestions.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 8, 2011 5:42 AM   in reply to Islanders66

    I've found most of solutions in the drive set up http://forums.adobe.com/message/3023501#3023501.

     

    With 6-8 disk one disk as OS and the rest in Raid 3 or 5.

     

    Maybe this thread was created before that one becaue most of the post here are about a 4 disk set up and not about raid. I also haven't found any info about the compatibility between Sata II and Sata III ports.

     

    I'll be installing 5.5 later this week and look forward to working with the program and sort out the set up issues and conflicts as best I can.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 8, 2011 11:49 PM   in reply to JEShort01

    Jim - You had asked:

     

    "I've  always been a bit curious about the Sans Digital TR8X unit that you are  going with, and I have a few questions - two of which you may not be  able to answer just yet:

    1) What connector / method are you using  to convert one of your internal SFF-8087 connectors to work with the  TR8X's SFF-8088 cables?

    2) How quiet / noisy is it?

    3) What are the max. temps. tlhat your 2TB's see when working hard; say, after a 1 hour render?"

     

    Answers:

    1) SFF-8087 to SFF-8088 cables from PC PitStop, not the adapters.

    2) It's a bit noisier than the Mac Pro itself at idle, but a lot quieter than the Mac Pro when all its fans kick up. I like how quiet it is.

    3) Toward the end of hours of constant disk activity (due to a Check Volume Set operation) the highest disk temp was 42 deg C. The ambient room temperature was 80 deg F indicated on a wall-mounted thermometer. Here's a RAID hardware monitor dump:

     

    CPU Temperature59 ºC
    Controller Temp.43 ºC
    12V12.038 V
    5V5.053 V
    3.3V3.312 V
    DDR-II +1.8V1.840 V
    CPU    +1.8V1.856 V
    CPU    +1.2V1.264 V
    CPU    +1.0V1.056 V
    DDR-II +0.9V0.928 V
    Battery StatusCharged(100%)

    Here's that hottest drive status (it dropped one degree):

    Device TypeSATA(5001B4DB0091E01E)
    Device LocationEnclosure#2 SLOT 03
    Model NameWDC WD2003FYYS-02W0B0
    Serial NumberWD-WMAY02014542
    Firmware Rev.01.01D01
    Disk Capacity2000.4GB
    Current SATA ModeSATA300+NCQ(Depth32)
    Supported SATA ModeSATA300+NCQ(Depth32)
    Error Recovery Control (Read/Write)7.0/7.0 Seconds
    Disk APM SupportYes
    Device StateNormal
    Timeout Count0
    Media Error Count0
    Device Temperature41 ºC
    SMART Read Error Rate200(51)
    SMART Spinup Time253(21)
    SMART Reallocation Count200(140)
    SMART Seek Error Rate200(0)
    SMART Spinup Retries100(0)
    SMART Calibration Retries100(0)
     
    |
    Mark as:
  • Currently Being Moderated
    Aug 9, 2011 4:20 PM   in reply to wonderspark

    Today is hotter in my office, and the hottest drive was 50 degrees C.

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 27, 2011 3:54 PM   in reply to Harm Millaard

    Is it a bad idea to use hard drives that are the same make and model but purchased a few months apart for a raid 0 or 5 array?

     

    I've got 3 Samsung HD103SJ's and I'm thinking about getting a few more, plus a 150 gb velociraptor, to experiment with some

    software raids before i take the dive into some hardware raid.

     

    Thanks.

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 9, 2011 2:22 AM   in reply to wolverine.ks

    No, so long as they're the same make and model. I have 8x WD2003FYYS disks in an 8-member RAID6 that I bought over a month or two spread from different vendors. They run great!

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 23, 2012 3:18 PM   in reply to Harm Millaard

    I'm potentially buying 5 drives, one ssd and 4hdd's. if i have esata drives for backups, at the end of the day, would this be feasible in the needs of price and speed:

     

        -(C:) 256GB Samsung 830 - OS, Programs

     

        -(D:) 2 Samsung 1tb Spinpoint Raid0 - Media, Projects, Pagefile

     

        -(E:) 2 Samsung 1tb Spinpoint Raid0 - Previews, Media Cache, Exports

     

    Or maybe just make (D:) a Seagate Cheetah for the peace of mind and added speed. Considering my renders wont be "just right" the first time for drive (E:), raid 0 for (E:) and backups at the end of the day gives me the benifit of added speed and less redundancy in my motions as oppossed to (D:) which are most likely gonna be solid across the board w/ my Media.

     

     

    I hope that makes sense, thanks for any help!

     

    -Lyndon Alvarez

     
    |
    Mark as:
1 2 3 Previous Next
Actions

More Like This

  • Retrieving data ...

Bookmarked By (2)

Answers + Points = Status

  • 10 points awarded for Correct Answers
  • 5 points awarded for Helpful Answers
  • 10,000+ points
  • 1,001-10,000 points
  • 501-1,000 points
  • 5-500 points