Skip navigation

To RAID or not to RAID, that is the question

Nov 16, 2009 3:46 PM

  Latest reply: Lyndon1984, Jul 23, 2012 3:18 PM
Replies 1 2 3 Previous Next
  • Currently Being Moderated
    Mar 24, 2010 8:22 AM   in reply to Harm Millaard

    I had a couple of quick questions regarding RAID and best disk set up on a PC for my requirements, if anybody had the time to help......

     

     

    1st question: I mainly edit 3 - 4 minute video shorts to music.  I film on an EOS7d and a Sony Camcorder (HDV). I do this as a hobby not professionally or for pay.  My main requirement for RAID is a belt and braces approach for ensuring that my original video files and photos never get lost.  I've just had a NAS expire that wasn't RAID so was considering replacing it with one that was.

    Given that I only do this for fun - I was thinking of storing all my video and photo content on a NAS with four 2TB disks (RAID 1) and then copying across to my PC only the files that I need to perform an edit.  I've seen the Synology 410j which appears like it might do the trick.

     

    If I then lose my work in progress on the PC - it is no big deal (annoying - but as I'm doing this as a hobby I could live with losing WIP as long as my original files are always protected).

     

    Does this sound like a sensible thing to do - given I don't really think I need the cost of a dedicated RAID card on a PC and I will use the NAS for other things - like music for my Sonos system etc?

     

     

    2nd question:  I have a reasonable spec PC - but think I made a mistake with my hard disk set up.   Basically, if all my content is protected on my NAS, I just want the fastest performance for editing and from the built in disks on my PC.  Clearly if I can get RAID cheaply on the PC then great - but speed and stability is most important.

     

    I currently have 1 x boot drive with a 250GB SERIAL ATA 3-Gb/s HARD DRIVE WITH 8MB CACHE.

    I then have 2 x 2TB SERIAL ATA 3-Gb/s HARD DRIVE WITH 32MB CACHE in RAID 0 (stripe)

     

    Basically I'd like to speed up the editing process on the PC.  I have i7 920, 12GB CORSAIR XMS3 TRI-DDR3 1333MHz, ASUS® P6T WS PROFESSIONAL, 2GB NVIDIA GEFORCE GTX285 PCI EXPRESS, Windows 7 Professional 64 Bit  and the hard disks I mentioned above.

     

    The maximum size of content for an edit consists of 80gb worth of Cineform files.  So if I copy that from a NAS to the PC I can then start an edit.  (I use Adobe Creative Suite 4.1)

     

    What disk configuration should I get on my PC?

    How should I structure them?

    What files should I put where?

     

    Happy to pay for small SSDs for boot drive etc if that makes a difference?

    Or perhaps Velociraptors?

     

     

    The main reason for making the editing as quick as possible is that I have limited time.  This is a hobby and I have a real job so need to get edits done as quickly as possible.

     

    Thanks in advance if anybody has the time to advise / help.

     
    |
    Mark as:
  • Currently Being Moderated
    Apr 24, 2010 7:05 AM   in reply to Harm Millaard

    Well Thankyou Harm, that is the most useful info on RAID I have found on the net. Unfortunately the net also led me to a page where someone stated that 'editing HD movies in APP is futile without RAID'. Reading through everyone's posts on here, this is obviously an untrue or old statement so I'm looking to find some trustworthy advice on the best setup for editing my daughter's HD .mov's shot on a Kodak V1273 at 1280 x 720. Data Rate 7.14 mbits/sec. I currently have:

    Mobo: AsRock 4Core1333 viiv

    CPU:  LGA775 Intel Pentium Dual Core E6300 (2.80 stock, it's at 3.6 atm)

    RAM: 1GB Ballistix Tracer, 1GB GEIL 667 (DDR2)

    GPU:  Sapphire Vapor-x HD5770 1GB

    HDD:  2xWDC WD1600JS-75NCB2 (Western Digital 160gb, one has had a lot of use and is slower and noisier than the other)

              1xST3808110AS (Seagate 80GB)

     

    This is obviously inadequate to edit HD movs and I am looking to upgrade next month. I only have the trial version of APP 4.0 CS4 and I will be purchasing a licence key and Windows 7 Ultimate 32-bit with my (son's!) new build.

    I am looking at upgrading the MOBO, RAM and HDD's to:

    Mobo:   ASUS P5Q Deluxe P45                      £129.99 GBP

    Ram:    PC2-8500+ OCZ Reaper (2x2GB)         £ 99.99

    HDD's:  4x WDC Black 640GB 6GB/s              £ 54.99 each so I think £119.96?

     

    My must have's for Mobo are: LGA775 (I don't have the money go i5  or i7), e-SATA and good o.c.-ing abilities/stability. The ASUS has these.

    I don't have the money to move onto i7 yet and even i5 boards with SATA 6GB/s connections are too pricey with the CPU.

    My budget is only about £ 450.

     

    I think the question is:

    1. Does anyone know if there is / should I wait for a board that's LGA775 DDR2 e-SATA (e-SATA is a must) with 6GB/s SATA or would my upgraded system above without 6GB/s be sufficient when set out right?

    2. If the above system is fine, would I be better or worse off RAID-ing for performance?

     

    Thankyou Everyone your input will be greatly appreciated.

     
    |
    Mark as:
  • Currently Being Moderated
    Apr 24, 2010 8:24 AM   in reply to Harm Millaard

    Harm,

    Thankyou for your quick response.

     

    Judgding by your reply to mine and other posts, 1) I am understanding that the best way forward would be to go for the upgraded system and forget about SATA 6gb/s until I can afford i7 end of this year. I need to upgrade my Mobo as the one I have is very unstable. 2) I am also understanding the best way to set up my HDD's would be:

     

    C: 640 GB for just OS &  Programs

    D: 640 GB for just pagefile,  media cache and previews

    E: 640 GB for just project save files,  indexed and conformed/peak files

    F: 640 GB for just original media

    And of course change Win7Ultx86 for Win7Prox64

     

    3) I'm understanding that a big thing is for these hard drives to be empty and not full up of my backups: I have a 500 GB external  and a 500 GB WDC Black coming for external backups. I'll only have the time to use the computer a few hours every week but (external) backup is/has always been big for me.

    Are my understandings correct?

     

    Thankyou again Harm, I know that you do this for free and in your own time and I really appreciate your help.

     

    Simon

     
    |
    Mark as:
  • Currently Being Moderated
    Apr 24, 2010 9:45 AM   in reply to Harm Millaard

    Harm,

    That's great thankyou very much

     

    Simon

     
    |
    Mark as:
  • Currently Being Moderated
    Apr 26, 2010 8:56 AM   in reply to Harm Millaard

    Excellent explanation. A few things to consider now are the changes that the tapeless workflow has brought to video. Before the tapeless codecs backing up video raws and material was really based on production time and environment more than anything. The enterprise world had to maintain the type of data recovery for ages while the video world could just go back to tape. Now video has entered the enterprise world with the amount of data to backup or maintain and the length of archive. I have many clients who have to maintain raws and or at least finished projects for 6 months to a year period of time. A standard project could be 56GB to 100GB of data that has to be stored with the current codecs. What that means is redundant arrays are for more suitable now to the video world as they have always been for the Enterprise community.

     

    Raid 6 has one big advantage over raid 5 or raid 3. The read speed is significantly faster than raid 5 and the redundancy is far more secure than raid 3. The only requirement is as Harm said the hardware controller. In the tapeless world, editors are not capturing anymore. They are simply copying files from one media to another. This limits the write draw back to raid 6 due to the sequential write that happens most of the time with this method and the higher access time that raid 6 has is negligible when copying few large files from another media that is far slower than the raid anyway. What you need in a tapeless workflow is read speed during the edit workflow which raid 6 has in abundance. The render time writing files is based on the codec and often does not have much performance gain when the write is over 150 to 200 MB/s. You also have the best level of data protection allowing up to 2 disks to fail simultaneously and still retain the array in degraded status. If you keep a hot spare available then your chance of complete failure is near minimized to a complete environmental failure such as a power hit. That can be also limited by software backups to a NAS or E-Sata drive that is only connected to power when needed.

     

    The raid controller is the component though that decides the level of performance and data protection you get. Areca is one of many controller manufacturers that you can get. The performance hit over 50% capacity Sata drives have also happens in raid arrays depending on the controller and the level of raid. Many of the $400 to $600 raid controllers will give you full performance throughout the entire raid volume in a Raid 0 but will still get the 50% volume capacity performance drop with Raid 5 or 6. That also changes when manufacturers update their firmware and change their controller cpu's. The Atto controllers use to be able to maintain performance over the entire array in raid 5 or raid 6 arrays but now have the performance drop with their current chips and firmware. Areca's mid range controllers have always had that performance hit but evidently their higher end controllers that harm has do not. The best controllers you can get are always SAS controllers. Those are basically SCSI controllers which handle Sata drives. SCSI controllers were always able to maintain the performance in an array which was why they were so dominant a decade ago. Now they use the same controller type with Sata drives. The reason some are able to maintain performance and some are not depending on the raid level has to do with the firmware and how it communicates with the controller on each HDD itself. That is really what you are paying for besides the extra ram when buying the $1000+ controllers. Also the raid management software and driver has some affect on this as well. The raid management software affects how efficient and accurate your redundancy is maintained and your rebuild success. It also can decide if your raid controller can get confused in a few scenarios and corrupt your raid 5. I have not seen a raid 6 get corrupted yet so not sure if that can happen. Either way I use and push my client's toward Intel's SAS controllers. They have the new SAS 600 models and they perform as good as Areca or Atto since they all use LSI chips. They have the best management software out that I have seen and they are normally half the price of ATTO controllers. Just some further information to help with deciding what is best for your situation and editing.

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 19, 2010 4:20 AM   in reply to Harm Millaard

    Hi,

     

    I followed your advices and configured my media system with WD Black 2TB disks, but the data transfer seems still slow...

     

    As for the other it worked well with your configuration and macbook pro, I have to conclude that the problem is the external case -

    I use Raidsonic dual bay with USB 2.0

     

    Can you recommend any other external case, I could use (2-4 bay) with FireWire 800 (as eSata to Macbook does not work to my understanding)

     

    Cheers!

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 19, 2010 5:35 AM   in reply to Harm Millaard

    Harm, just a comment on that particular 4 TB external drive with the two big Hitachi's . I have a new one on a workstation (win 7 x64)  with the eSata connection and am not real impressed so far. It tested at 160 MB/s with HD Tach, but in real life seems to be much slower. Transfers to it from another internal 1 TB SATA drive were only 64 MB/s, which is much slower than going from my other internal drives to each other, or across our 1 Gb network.

     

    I'm not sure if something else is wrong somewhere in this setup, but he probably should get other opinions before deciding. If it's something else on this particular ws, then it still could be a very good solution. But it would be good to hear from others on that drive too...

     
    |
    Mark as:
  • Currently Being Moderated
    Jul 19, 2010 8:04 AM   in reply to baumanico

    baumanico wrote:

     

    Hi,

     

    I followed your advices and configured my media system with WD Black 2TB disks, but the data transfer seems still slow...

     

    As for the other it worked well with your configuration and macbook pro, I have to conclude that the problem is the external case -

    I use Raidsonic dual bay with USB 2.0

     

    Can you recommend any other external case, I could use (2-4 bay) with FireWire 800 (as eSata to Macbook does not work to my understanding)

     

    Cheers!

    Try looking at this one as an example

     

    Here is a great vendor that caters to Mac but they only sell units with drives in place.

     
    |
    Mark as:
  • Currently Being Moderated
    Calculating status...
    Aug 1, 2010 1:54 AM   in reply to Harm Millaard

    I didn't want to make another topic so decided I'll make my try here:

     

    At the moment the only disk I have is WD Caviar 160GB and some WD Passport 500GB for backup. I was wondering which option would be better to speed up my computer a little:

     

    SSD 30GB for cache files or 2x500GB RAID0 for cache files?

     

    Or maybe the best option would be to buy these 2 500GB WD Black and then:

     

    C: 160GB for OS and programs

    D: 500GB for pagefile, media cache

    E: 500GB for project files/all other media

    F: 500GB G-Drive e-Sata for Backup (which I want to buy too)

     

    I'd be very thankful for all your answers.

     

    Regards

    Paul

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 1, 2010 3:15 AM   in reply to Harm Millaard

    Thank You very much Harm

    Now the only consider is - WD Caviar BLACK or Samsung Spinpoint F3. I must do some research

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 1, 2010 3:50 AM   in reply to Harm Millaard

    Harm, I agree very much with your layout and we followed your suggestions a while back and went to single drives on our  3 workstations and have had much better stability and peace of mind. However, as much as we would have loved to have put an Areca RAID card in each ws for our backup, our non-profit budget just would not handle it.

     

    I have opted for another cheap solution that could be a good tradeoff  for others too, and would like to hear your comments on it:

     

    We really did not want to use more than four TB drives in each ws (except in our CS5 ws), so opted for having just a boot/prog drive, with another drive for data and the rest. We have been using the other two TB drives as mirrors (RAID1), through a software solution (MirrorFolder) and have had well over a half a year of stability and satisfaction. When one of the OS drives failed, MirrorFolder automatically kicked in at reboot and allowed us to boot off the OS mirror and install another drive very quickly. We lost almost no time and no work.

     

    I realize this is a tradeoff and the ideal would be for the hardware to handle the RAID1, but am offering it as a very nice and cheap alternative to other ways. It has given us great peace of mind and real confidence, especially when backing up to an external 4 TB eSata every now and then. We take a slight performance hit under Win7 x64, but nothing that was enough to bring CS3, CS4 or CS5 to their knees yet.The newest version of MirrorFolder (5,0,252) is very nice and leverages multicores and is quite stable under Win 7 x64. We love being able to mirror an occasional odd project folder at any time with it, too.

     

    Comments?

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 1, 2010 4:43 AM   in reply to Harm Millaard

    Maybe someone would be willing to do a "low end" test on the difference between the two? Since this is about the cheapest and least time consuming (from a backup standpoint) of all the "security" solutions, I'm sure there might be a number of others interested in this as well. The new feature set of MirrorFolder is pretty complete, so it could be revealing to those of us with built-in RAID chipsets (but using software solutions instead). MF is especially easy for adding drives later on.

     

    I forgot to mention that MirrorFolder 5 is a beta version I think, so it might be a bit more difficult to obtain for testing without a license.

    ******************

    After checking, this was in the email TechSoft that sent out:

     

    MirrorFolder v5.0 (beta) is now available for download and testing from  the link given below. You may use your existing v4.x serial number to  install it freshly or over your existing MF v4.x. You may also use the  temporary serial number "XXXXXXXXXXXXXXX" (two computers)  to install it on maximum two different computers for evaluation.  However, this temporary serial number will expire on 5th Oct, 2010.

     

    To save you work if you wanted to know what is different from the past, here's their info on what's new:Here are major changes in MF v5.0:

    * Added support for copying open files using Windows Volume Shadow Copy Service APIs in all synchronization modes.

    * Added option to perform synchronization in a separate thread with different priority.

    * Added option to perform multi-threaded synchronization on multi core/CPU systems.

    * Added option to send email notification on error/warning, and successful synchronization/archive operation.

    * The main user interface program now shows volume label of  source/mirror drives that are not currently connected. But it still uses  volume ID to identify source/mirror drives.

    * Supports for selection of network drive/folder as source to mirror on another network or local drive.

    * Added command line option to change volume for the mirror folder to mirror it on a new drive.

    * Added option to encrypt ZIP files.

    * It now automatically use 64-bit ZIP file format if ZIP file size grows beyond 2 GB.

    * Added option to synchronize when user remains idle for the specified time.

    * Support for Windows 95, 98, Me, NT 4.0, and 2000 has been removed.
             
    * Many other small changes and improvements.

     

    ***************

    I'd love to see someone test this, as it might save them money (or convince them to buy something useful and cheap that is a real help and very easy to use).

     

    Message was edited by: sarmour2 - to add information

     

    Message was edited by: sarmour2 - removed temp serial number

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 16, 2010 9:46 PM   in reply to sarmour2

    Good question

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 16, 2010 9:48 PM   in reply to Harm Millaard

    Great posts and some great answers out there. I am just getting my NLE machine up and running and will be doing 5-10 min. segments shot with Canon 5DII in full HD. I have read most of haarm's threads as well as many other resources on RAID 0. Against some of the recommendations I have purchased 2 WD 1 TB Black drives that I want to set up in RAID 0. I will mostly be using PP and I read on Adobe's sys req that you needed RAID 0 for uncompressed.

    -ASUS x58 Sabertooth

    -I7 950 with mild OC 3.50

    -12gb Ram

    -Seagate 1 TB (os) sata

    -2x 1 TB WD Black Caviar (I want to raid 0) sata

    -2x optical drives sata

     

    I can not for the life of me get the RAID to work with the ICHY10 mobo controller. Here are the steps I am following:

    Boot to BIOS set sata to RAID 0

    Go in to Matrix Drive Manager Make Raid Volume out of 2x WD 1 TB

    Boot to Vista 64 disk

    In disk manager of install I have the following;

      DISC 0 1864 GB unspecified data

      DISC 1  936 GB Main

    I select the 1864, hit new, specify the size, format, than I get this:

      DISC 0 1864 GB unspecified data

      DISC 1  936 GB Main

    I hit the install drivers link in the bottom left and load the ITCHY10 controller drivers from the mobo disc then put the windows disk back in and install too the smaller 1 TB drive. When windows reboots it hangs at the bios screen when it gets too SATA 4 every time for 30 seconds then it goes on to the windows screen with the scrolling green bar and freezes there. Any ideas? Do I even need RAID 0 or will it work using Haarm's 3 disc set up? Remember it is all HD video I will be editing but not many tracks just lots of cuts.

    Thanks in advance.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 16, 2010 10:12 PM   in reply to mcshorty

    So you lose your sunglasses and look for them for an hour, finally in desperation you reach out to your friends and ask them if they have seen your sunglasses. You notice as you say this they are starring at your head not your eyes. Your hand quickly moves up to feel the top of your head and bumps into the "lost sunglasses"

     

    Well no sooner did I ask this last question, I unplug the two RAID 0 drives and do a hard reboot as it was frozen. The boot up is still slow but different. It makes it through to the windows screen and says finishing installing windows. Maybe I wasn't patient enough the other 10 times or maybe it was that I unplugged the 2 RAID drives, or a mixture of both.

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 17, 2010 6:33 AM   in reply to mcshorty

    During the post, you have the drive detection and smart requests. If a drive does not respond initially then the bios will continue to send the requests until the timeout period has been exhausted. That time out period is normally set to 30 seconds by default in the bios. You can lower than to 5 or 10 seconds if you want to. However what this means is you have a drive, likely number 4 that is not responding to the bios. You need to check the cables to that drive and try it on a different sata port. You might have a bad drive.

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 25, 2010 6:37 AM   in reply to Harm Millaard

    "Harm" says: ....

    "RAID10

     

    The RAID level for paranoids in a hurry. It delivers the same redundancy as RAID 1, but since it is a multilevel RAID, combined with a RAID0, delivers twice the performance of a single disk at four times the cost, apart from the controller. The main advantage is that you can have two disk failures at the same time without losing data, but what are the chances of that happening?"

    -----------------------------

    Uhmm:

     

    Raid 10 deserves a little more love,

     

    Raid 3 is impratical for "MOST" users even with Hardware Raid controllers, unless like you said, you get a mortgage to buy Areca,...

    -> Any "parity"-based Raid lvl, such as 3,5,6,50,60,... can NOT compare to the straight out "sustained" performance of the multilevel-"KISS" Raid10 (aka 0+1, 1+0).

    Actually, "Parity" Raid levels historically, were not meant for "great" performance, as much as they were designed for massive storage solutions. -(ya know, back in the day when disk storage was NOT cheap !).

     

    -> ie. with an 8-Disk Raid10, now you will accurately get 4x the performance of a single disk at only twice the cost. (4-Disks x 2)

     

    Everyone knows Raid10 is the best Raid for "performance" -especially when you need to consider "SUSTAINED" writing,(aka monstrously huge 1080p-style video files, ...).

     

    -> The other obvious thing to consider is that Raid10 does not suffer from performance degradation when it needs to rebuild the array, in fact, Raid10 is virtually instantaneous, unlike the "parity"-based Raid's (aka 3,5, 6...) which can take literally hours, or even days to Build\rebuild while suffering from read/write performance degradation  while it's re-building.

     

    The only disadvantage to Raid10, and I do mean the only one, is that you have to consider purchasing twice the capacity required, but Raid10's resilience, performance, and redundancy far outweigh the initial cost of the disks, especially considering the cost of Disk space today ?! comon !

     

    For example, I recently bought (some el-cheapo) (8) Hitachi Deskstar SATA2(7200rpm) 2-Terabyte per/disk for just over $100/each.

    In a Hardware Raid10 configuration I easily get ~600 MBytes/sec read/write, (using BlackMagic's Disk Speed Test Utility), and that ain't bad. I also could lose up to half my Disks without worrying about any performance hit whatsoever, at any time.

     

    -> Lastly, "ANY", and "ALL"  decent hardware RAID contoller's have at the least, a Raid 10, <- It is FUTURE proof.

     

    In other words, no I do not use Areca, but even if I did, I would use Raid 10 for the obvious reasons.

     

    But also, other hardware Raid controllers, such as Adaptec, or  LSI/MegaRAID/3ware, ....can also fit most Video-editing users' needs at half-the price of that Areca.

    Either way, Raid 10 should be given primary consideration, before having to consider Raid 3, 5, 6 ...

     

    Cheers,

     

    Rick.

    Attachments:
     
    |
    Mark as:
  • Currently Being Moderated
    Nov 28, 2010 5:54 AM   in reply to Harm Millaard

    Great thread(s) on RAID!

     

    But, where and how do I set-up all my Adobe appz to use my RAID0?

     

    Should I mount it as a folder, like /data? or as D: / ?

     

    When using Premier with ONE HD for boot and appz and data, even if a 15K SAS, its so slow and frustrating!

     

    I have 4 x SAS 15k Hitachi HDs for use in a RAID system and have them in a RAID0 at present as they are only 75GB each in size, but I will use RAID3 if it's a much better path.  suggestions?

     

    Can I set-up ALL my Adobe appz at once to use this RAID sub-system?  reguardless, where do I do this within the adobe appz?

     

    thanks,

    billy

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 28, 2010 11:07 AM   in reply to Harm Millaard

    I understand. I must not have explained my question well. This is an old thread I thought may would help me out some. It's not have orignal thread.

     

    I already have built my RAID0 (but could change to RAID3) and use Highpoint's RocketRaid 4320 hardware controller. My question has to do with how best to attach it (or any HD) to improve my video editing performance. I can attach the HD as a separate drive as in D: \ or I can mount it as a folder such as \data under C: . I am not sure which way would boost performance for editing video such as with Premier or Director.

     

    Anyways, I was hoping someone could comment on this concern.

     

    Thanks,

    billy

     
    |
    Mark as:
  • Currently Being Moderated
    Nov 29, 2010 11:35 AM   in reply to RoboGuighes

    You might want to check raid 5 and raid 6 performance results again with a standard enterprise level raid controller because things have changed. An 8 drive 7200 raid 5 array on a standard controller like the Intel raid controllers are sustaining 650Mb/s to 700MB/s depending on the drives used for both read and write. Raid 6 on the same controller is averaging 635MB/s to 650MB/s Read and 600MB/s Write with the same 8 drives. You are definitely accurate with the rebuild times although raid 5 and raid 6 rebuild now is down to an hour or 2 with the same controllers. The load is no longer an issue during rebuild as well due to background rebuild options currently. The benefit of raid 10 is the ability to run those arrays on cheaper controllers such as the onboard without having the rebuild penalty that you get running a parity raid. Also the integrity of the redundancy is not as good with parity raids on the onboard controller. However the money you save on the controller, you pay with the extra disks for a similar volume size. Add to that the fact it matters which disks fail with a raid 10 when 2 fail versus any 2 disks in a raid 6 and the old mantra of a raid 10 is no longer the case. That is why we propose parity raids versus raid 10.

     

    Eric
    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 13, 2010 8:31 AM   in reply to Harm Millaard

    Excellent post.  I have learned much from this post.  I have a much better appreciation of what I am planning to implement.  I guess I am "paranoid on speed" as I plan on 1+0.  Here is what I have planned:

     

    OS drive:  Western Digital VelociRaptor WD6000HLHX 600GB 10000 RPM 32MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive.  I will be running Windows 7 ultimate 64 bit, not pro as recommended, as I purchased this at a very discounted rate.

     

    Working drives:  SAMSUNG Spinpoint F3R HE103SJ 1TB 7200 RPM 32MB Cache SATA 3.0Gb/s 3.5" Internal Hard Drive -Bare Drive.  I originally went with WD black, but read many issues with TLER.  As such, I will be using the Spinpoint f3R (not F3) - I will pay a few dollars more for the higher grade.  These will be configured in raid 1+0.  Any issues with Samsung F3R running in Raid 1+0?  I did not go with the WD RE4 due to the cost increase vs. Spinpoint.

     

    The 1+0 will done using the MOBO (MSI XPower Big Bang) as now I understand there is not much advantage in a separate hardware controller for my situation.  Here is what I do not quite understand yet after reading all the posts.  As I will be running raid 1+0 on the 4 "working / storage" drives, not the OS drive, is all the "configuration" data stored in the BIOS, or is some stored in the OS?  As I am not planning on using any redundancy Raid on the OS drive, will this cause issues if my OS drive crashes and needs to be rebuilt?  Will it lose pertinent data needed to keep the raid integrity intact?  I will be doing an Acronis backup monthly on the OS drive however.  Will this be adequate?

     

    Thanks in advance.

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 16, 2010 3:47 PM   in reply to sldwaa

    The raid MetaData is stored on the drives in the volume. This is how you can migrate raids from 1 system to another even changing controllers sometimes. Although the changing controller part is sketchy and may not work. It depends what chip the previous controller was using and the drivers for the new raid controller. They have to allow importing foreign configurations. So to answer your question, the raids will be fine if the OS fails.

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 16, 2010 4:45 PM   in reply to ECBowen

    Thank you - I am going to move forward with the 10 as described.

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 17, 2010 1:05 PM   in reply to Harm Millaard

    hi harm

     

    could you tell me what you think about my configuration and the way i use it ive got 4 hard disk ?

     

    1) wd 2,5size  velociraptor 300 gb 10000rpm partion in 2 (150gb for win 7 and program and 150gb for archivie)

     

    2) wd 500  gb 7200rpm that i use only  for premiere project whit video file that i've import

     

    3) wd 500 gb 7200rpm that i use for rendering of the premiere project

     

    4(wd 1t b7200 rpm that i've partioned in 2 (500gb for archivie file music, photo, ecc and another 500gb i use for different file word ecc)

     

     

    Dou you think it's usefull put premiere project in one hard disk and the rendering in another one or is better to put them in raid 0 ?

     

    or there is a best way to use my drive?   maybe i have to create a little partion of the 1tb and use it for file pagging?

     

    after tomorrow i'll format my pc and re-installing win 7 and adobe suite so it will be an opportunity for me to change my configuration

     

    thank

    silvio

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 21, 2010 5:10 PM   in reply to Harm Millaard

    hello,

     

    i just posted in PPro cs5 and it got bumped to the 'hardware' forum

     

    and so i've read this thread and noticed Harm touching on 'cluster size'

     

    which is the subject of my post...

     

    for HDD (not SSD)  what is the best cluster size for non OS disks, non pagefile disks?

     

    For disks like video assets and audio files and final renders

    wouldn't a larger cluster size result in faster/ more efficient performance?

     

    thank you in advance,

     

    jeffrey

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 22, 2010 4:34 PM   in reply to Harm Millaard

    Looking for recommendations on RAID set up:

    I have CS5 running on an I7 OC to 3.3 mhz with 12 gb of RAM, Vista 64. I just lost one HDD out of my three drive set up. 1= 1TB non-raid OS and programs 2=1TB raid 0 vol.1 3=1TB raid 0 vol.2 (I know raid 0 is bad) I had minimal loss of data but now I am ready to take the next step. I usually only edit about 3-10 minute shorts all in HD from the canon 5D mkII. This usually translates to 60 minutes of HD footage and 100 stills needing to be used at a time for any project. I am looking for a cheap but fast and secure way to store my projects. I am thinking about an areca or adaptec controller with 4 disks in RAID 10 in an external enclosure that could be updated later. I would be able to move all projects and file to a single non raid external/ NAS after completion. I went with RAID 10 because of the rebuild time and redundancy yet it will offer me the speed of my RAID 0 set up that failed. My questions are on how to best implement this solution.

     

    1. What kind of actual I/O speed do I need for editing in full resolution? Short projects so render times and read write were never an issue with RAID 0.

     

    2. How does the External Attached Storage work? i.e. external box has it's own controller or do I need a controller in the WS as well or can I just plug it straight to the eSATA port on my ASUS Sabertooth X58 mobo?  I am getting confused by the port multipliers and the HBA's described on the ARECA site.

     

    Thanks in advance for the forums help..

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 29, 2010 8:15 AM   in reply to Harm Millaard

    You wrote: "Motherboards often have some limited raid capability  with the ICHR10 chip. That is a perfectly feasible solution for either  raid0 or raid1."  So for a moment I thought: 'not for Raid10'. Then I red your start: "For RAID levels without parity check like Raid0, Raid1 and Raid10 software controllers are quite good with a fast PC." So this PiaH is happy. Furthermore, googling on the string "RAID 10 on ICH10R" did not result in a lot of troubles, on the contrary, it seems to work well, the most critical remark I found was at a 6disk RAID 10 on ICH10R: "It will work, but you will not see the full potential out of the raid array without a dedicated hardware based raid card."

     

    I  would like to ask you the following, given the fact 1) the building business where I operate is extremely bad and my budget is very tight after buying ASRockX58Extreme6, Intel-i7-950, 3x4GB KHR1600, Seasonic750W, C300-SSD forOS/Programs and Win7Ult and 2) I am mainly working with Photoshop but  sometimes want to touch video-editing to broaden my market:

    I'm  opting for the 4-disk config from your Guideline with E: and F: in  raid0. This is actually your 3-disk config with the Pagefile on the  E:, isn't it? Then I would like to set up my D: (media, projects) as a  4 disk RAID 10 on ICH10R. With 4x 1T-SpinpointF3-disks of €50 (or 4x  320GB-spinpointsF4 of €35), this will only cost €200 for 2TB (or  €140  for 640GB). Given the fact that I am a poor Paranoïd in a Hurry, does  this make some sense, and can I even make this crazier with a 6diskRAID10? Or did I overlook something

     

    One complication. ECBowen wrote: "Add  to that the fact it matters which disks fail with a raid 10 when 2 fail  versus any 2 disks in a raid 6 and the old mantra of a raid 10 is no  longer the case. That is why we propose parity raids versus raid 10." Although I do not understand this, does this also apply to 1 failing disk?

     
    |
    Mark as:
  • Currently Being Moderated
    Dec 29, 2010 9:22 AM   in reply to Draal Lala

    With Raid 10, It will not matter which disk fails if only 1 drive fails. It's only if the second drive fails that it matters which one.

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Feb 3, 2011 2:04 PM   in reply to Harm Millaard

    Harm, I am looking forward to change my current disk set up because I have got lots of stock footage I have collected over the past four years. My current disk set is 1.5 tb x 2 in BIOS Raid 0 and 1.5tb x 2 in a TB 32 Raid 0 esata.

     

                                            1)     BIOS RAID 0 3TB partitioned into C & D

     

                                                                                                   C= 746GB (OS, program files, pagefile)

                                                                                                   D=2048 (Preview, Exports, Cache, projects)

     

                                           2)     Mapower TB32 3TB RAID 0 partitioned into E & F

     

                                                                                                   E=1800GB (Media)

                                                                                                   F=  900GB (Stock)

     

    I have got an offline 2TB which I regularly use to backup any content that I generate on my workstation, including all the stock footage.I have had this set up from December 2010 when I upgraded to Win 7 Pro. Since then I am having problems with the TB32. Firstly what happened is that the TB32 disappeared from windows explorer when I left my computer idle for some time. I looked for it in disk management only to discover the two partition appearing as unallocated, I was really surprised with this.I took it to my computer shop, who only told me that one of the drivers have failed. The drives are just over one year old, so I took them apart and tested them and formatted only to discover that they were working, but in the process lost all the data that was on the drives. I put them back again to raid 0 and deleted the partitions which appeared again despite the formatting. I then created new current partitions and everything started to waork again. A couple of days they were unallocated again, but this time I selected offline in disk management and then online and the partition came back again.

     

    Now the major problem is that when I am editing in Premiere CS5, if I leave it idle and come back after about some time the Premeire will crash and I have to relaunch it again, or the TB 32 will just go to sleep and disappear even in computer management which means I will have to switch it off and on again for it to come back to life. I never had these problems with Vista biz 64, so I am not sure what really is causing that. In vista I had two non raid internal HDDs.

     

    But anyway I have decided to move on and ditch the TB32 because it appears to be slow.. not sure but when I copy files from my internal raid the speeds ranges between 130Mbps t0 60 at times. Since my workstation HP XW6600 is limited to 2 3.5" drives I can change it to 3 2.5" then I can use 1 for C drive and then RAID 0 for the other two, maybe hardware if I get a decent card. Externally I will haul my current 4 1.5s tb to create RAID 5 with a total of 4.5tbs which I will partition appropriately. The next thing now is to get a decent Raid 5 enclosure and the one I can afford is listed HERE . A good enclosure which can oprotect my data in case of power cut is greatly appreciated, because I am always thinking that the TB32 might have suffered a power cut and corrupted my data.

     

    My system have got dual quad xeon (8 virtual cores) 24GB RAM and startech (silicon) card for esata and a MatroX RT X2 and GTX 285 2GB, Win 7 Pro and CS5 PP.

     
    |
    Mark as:
  • Currently Being Moderated
    Mar 7, 2011 12:34 AM   in reply to srukweza

    Hey Ham,


    Thanks for all your hard work and help on the forum. Great info. Now I come to you for some help and recommendation for performance/protection and backup solution. I am editing with Premiere Pro & After Effects CS5 using AVCHD from my GH2. I have a 27" iMac 2.8Ghz Intel Core i7 16GB Ram 1067 MHZ DDR3 ATI Radeon HD 4850.

     

    Currently I have:


    - a RAID 0 set up with a 2TB Lacie 2big Quadra on a FireWire 800 for editing video (Media Files, Scratch Disk)
    - a 1TB Lacie Rugged XL for Time Machine backup
    - a  2TB Internal Hitachi 7200RPM HD that stores my (OS, Applications, Personal Files, Project Files) with 400GB used up so far (plan to keep that to 1TB max since I've been told it bogs down your computer once you got past the half way mark of space)


    However, something doesn't feel right to me about this set up. Not sure what it is.

     

    I have been doing tons of research and I'm debating if I need to get something else. And came up with this list. Or even doing an OWC Turnkey Upgrade (http://eshop.macsales.com/shop/turnkey/iMac_2010_27/add_eSATA) and adding an eSTATA port. Do you think this is necessary for the type of editing I'm doing with AVCHD and I won't be doing 10 tracks.


    New list:


    120 GB OWC Mercury Extreme Pro SSD (OS & Applications) --> as my internal HD (OWC Turnkey Upgrade replacement)


    Elite-Al Pro Performance RAID 0+1 | 2.0TB Stripe+Mirror (2000GB+2000GB)  (Media/Projects & Previews/Exports)

    - and then once that is full I was thinking of swapping HD's and storing it manually with some internal drive cases (http://www.hudzee.com/)
    -  then use thisNewerTech Voyager Q- Quad Interface SATA 2.5" & 3.5"  Hard Drive Docking Solution to see what is on the backup drives later on


    Where would Pagefile and Media Cache go?

     

    Mercury Elite-Al Pro RAID SAFE 2TB (1TB x 2) 7200RPM 64MB (to back up personal files)


    I'm a little lost and confused. I just want something that offers speed, reliability and backup. Nothing over kill. Just to get the job done.

     
    |
    Mark as:
  • Currently Being Moderated
    Mar 17, 2011 10:04 PM   in reply to dreinz

    Can someone please help me out with my RAID/BACKUP solution and answer the above message posting. PLEASE! AND THANK YOU!

     
    |
    Mark as:
Actions

More Like This

  • Retrieving data ...

Bookmarked By (2)

Answers + Points = Status

  • 10 points awarded for Correct Answers
  • 5 points awarded for Helpful Answers
  • 10,000+ points
  • 1,001-10,000 points
  • 501-1,000 points
  • 5-500 points