5 Replies Latest reply on Jan 10, 2015 8:37 AM by cc_merchant

    20TB RAID

    barker85 Level 1

      Hi there,

       

      I am considering putting my media (almost 20TB of 4K RED footage) in a RAID0 and am intending to do this with 5x 4TB drives (with a software RAID due to budgetary constraints).

       

      I have never configured a RAID before and was wondering if my system will have any problems with this? (please find specs below)

       

      Additionally, I was wondering is it worth going for 7200rpm drives instead of 5400/5900rpm?

       

      Any thoughts are hugely appreciated. Many thanks...

       

      i7 4790k

      32GB RAM

      Nvidia 970

        • 1. Re: 20TB RAID
          cc_merchant Level 4

          VERY BAD IDEA!!! You don't care about these data? No problem if you lose them permanently? Do you have a back-up plan to store these 20 TB elsewhere?

           

          With this plan, you increase the risk of losing all data by a factor 5. If one disk fails, you lose all. Does not look like a sound plan to me.

           

          With 20 TB of data that are precious, that you can not afford to lose, you NEED at least a parity raid (3/5/6), which is not feasible on your platform, AND a backup.

          • 2. Re: 20TB RAID
            barker85 Level 1

            Yes, the data is backed up.

             

            I take it that RAID 3/5/6 require hardware controllers? Unfortunately there is no way I can afford to spend $1500+ on the products I have seen suggested but I'd be interested to hear if there are any cheaper alternatives. I have been using USB3, which mostly does the job but can be painfully slow, so I am looking for an improvement in speed that is substantial enough to avoid problems with lagging video.

            • 3. Re: 20TB RAID
              cc_merchant Level 4

              If you want to take the risk, go ahead with 5 x 7200 RPM 4TB disks in raid0. That will increase the transfer rate to around 500+ MB/s, but restoring those 20 TB in case of data loss will still cost you huge amounts of time from the backup. Your CPU may then be your main bottleneck with only 4 cores without overclocking.

              • 4. Re: 20TB RAID
                coonasty Level 1

                i dont think software raid is the way to go on this system (or any system for that matter, other than pure redundancy storage),  especially on an editing system where performance matters.  i dont think you'll get 500MB/s on a software raid.  also you'll be tying up the os, cpu, ram managing the array and take away from your editing.  being budget is a concern is there a chance to pick up a used areca for $100 off of ebay or cl?  your motherboard doesnt have a raid chip on it? i would figure out a hardware solution with parity for that amount of data.  you talking about forever to restore from a backup if 20tb crashes and burns

                • 5. Re: 20TB RAID
                  cc_merchant Level 4

                  Even if the OP would find an affordable Areca, his platform does not allow its use without dropping the video card from PCIe x16 down to PCIe x8, resulting in a 10 - 15% performance loss, because of lack of PCIe lanes. He may also lack the possibility to overclock his 4 core CPU, because of the PSU and possibly the BIOS of his system. For 4K material the CPU, clock speed and disk setup are the limiting factors. Due to budget constraints, I see no solution for him.

                   

                  You are correct that restoring 20 TB from a backup takes around 78 hours when restoring from USB3, certainly prohibitive considering the speed advantage of 5 disks in raid0, which will still be slower than a single SSD. Another factor not yet mentioned is the fill-rate of those 5 x 4 TB disks, which is getting very close to 100%, in which case the transfer rate on that software raid will drop down to around 200 MB/s with a CPU load of 8% or more.

                   

                  If budget allowed it, the only sensible solution to having 20 TB on-line on conventional HDD's is to use 12 x 4TB disks in raid3 with a HS or in raid6, which would result in a net capacity of 37.2 GB and a fill rate of 54% as a workable minimum.