21 Replies Latest reply on Dec 9, 2011 3:26 PM by wonderspark

    RAID3 Rebuild Times

    cfsKC Level 1

      Since recently installing an eight disk RAID3 with 1TB Samsung F3s, I have had to endure two rebuilds. Each has taken over 25 hours to complete. This seems to be an inordinaryily long time. I would appreciate any observations other RAID3 users might care to offer on the time required for any rebuilds they have had to endure.

       

      BTW: A major source of the problem which neccessited the rebuilds was an underpowered UPS. I mistakenly thought my APC 1500 was up to the job. I was clearly wrong and a CyberPower PP2200SW 1500Watt UPS unit is now on its way to my office.

       

      It's also worth noting that once again, Eric at ADK has come to the rescue to give this technological retard assistance in resolving the issue.

        • 1. Re: RAID3 Rebuild Times
          Jim_Simon Level 8

          That's strange that a UPS, which does not feed power directly to the RAID, would have any effect at all.  I'd have thought the power supply itself to be a more likely culprit in "underpowered" situations.

          • 2. Re: RAID3 Rebuild Times
            ECBowen Most Valuable Participant

            Remember the PSU only can feed what power is available. An APC 1500 can only give a load of 865Watts. When you have a 1200W PSU connected to a 865W Line Interactive UPS then the PSU can not initialize correctly at 1200W's. This then causes devices connected to the PSU not to receive the correct power or in the case of drives, not to spin up in time or at all. This the the biggest mistake made by consumers and professionals today. You must get a UPS unit with a Watts rating atleast equal and preferably greater than the Watts rating of every device plugged into it. Do not go by the model numbers because that is the VA rating and not the watts rating. The watts is what you have to go by.  Do not go by the precept that your system will normally not be drawing that load. Your system initializes at full power before it idles down and the hardware inside will not initialize correctly if you have a lower Watts rated UPS feeding your PSU and other devices plugged into it. Eventually you will blow your PSU and probably another component like the system board. This has become a very common miss conception today and is causing allot of problems. I get tech calls all the time over this because the client spoke to some idiot who said don't worry about it because it does not draw that much power when it's running. Ignore at your own loss.

             

            Eric

            ADK

            • 3. Re: RAID3 Rebuild Times
              Jeff Bellune Adobe Community Professional

              If the UPS doesn't have enough juice to match the power supply, then if

              the power supply demands more power than the UPS can provide you can

              certainly end up in an underpowered situation.

               

              Case in point: I had a 700-watt UPS and what I thought was an 800-watt

              PS.  I lost a memory stick due to unknown causes, and I can't rule out

              bad power since it turns out my PS is 1000W.  I now have a 1200-watt UPS

              and a new stick of memory.  Lesson learned.

               

              -Jeff

               

              EDIT: If the forum had been working properly today, I would have seen Eric's post and refrained from posting myself, especially since Eric helped me troubleshoot the bad memory.  But it wasn't and I did.  C'est la vie.

              • 4. Re: RAID3 Rebuild Times
                Jay Knobbe

                I copied this reply from ALLEXPERTS web site:

                 

                "The amperage available at any wall receptacle depends on the size of the circuit wiring and breaker...in most homes - #12 copper is run for the circuits...making 20 amps the maximum that circuit can safely handle...and you should stay well below that at around 16 amps max on a 20 amp circuit. However...some homes have #14 copper on some circuits and are rated for only 15 amps maximum...and you should then stay at 12 or less amps max on those circuits. Also...if aluminum wiring was used...you'll also need to stay down around 12 amps."

                 

                This statement would only be accurate in the USA I guess.

                 

                Also what must be taken into consideration is other loads on that circuit.

                 

                I also have a APC RS1500 rated @ 865w and am running a 1000w PSU so it looks like I am way under the threshold even tho only the PC is plugged into the UPS. Monitor, etc is plugged into another UPS, but on the same circuit.

                 

                So it looks like I'm calling the electrician today and budgeting for a new UPS.

                 

                This is certainly something to consider when planning your PC build since it can add unexpected costs to get adequate, clean power to the PC.

                 

                I do have a question ....

                 

                If I have a 1000w PSU I'm guessing a UPS rated at 1000w is just getting by? I supposed a higher rated UPS would be recommended? There is a huge jump in price going from a 865w to 1200-1500w UPS.

                • 5. Re: RAID3 Rebuild Times
                  cfsKC Level 1

                  Now . . . about rebuild times

                   

                  My eight 1TB disk RAID3 array with an Areca 1880ix-16 controller currently contains a mere 500GB of data yet it takes over 25 hours to complete a rebuild.

                   

                  1. Is this a "normal" rate for RAID3?
                  2. What factors affect rebuild time?
                  3. What have others experienced with RAID3, 5 & 6?
                  • 6. Re: RAID3 Rebuild Times
                    ECBowen Most Valuable Participant

                    http://www.ultraproducts.com/applications/SearchTools/item-details.asp?EdpNo=1815470&Sku=U LT33046

                     

                    This is the UPS I would recommend for a 1200W solution. Yes you want greater than 1000W if your PSU is 1000W.

                     

                    BTW a 16TB raid 5 array on the Intel controller takes 2 to 5 hours to rebuild based on the current activity on the system. Remember many of the rebuild time complaints you read about have to do with people using the onboard raid for raid 5. The onboard raid takes 24 to 35 hours to rebuild on average.

                     

                    Eric

                    ADK

                    1 person found this helpful
                    • 7. Re: RAID3 Rebuild Times
                      Harm Millaard Level 7

                      I have had one disk in my raid30 array failing a couple of times, before I exchanged it. At first I could not figure out why my system appeared sluggish, till I figured out that the array was rebuilding. Then at boot time I got the message that the array was degraded and was in the process of rebuilding. I then decided I needed another disk, so I exchanged it and another run with rebuilding.

                       

                      This was on a 12 x 1 TB disk raid30 and I have not really timed it, and it is more than 6 months ago, but in my recollection it took around 5 - 7 hours. Mind you, this may be different than the figures you mentioned, because I have 2 GB cache memory on board of the Areca instead of the standard 512 MB, and maybe that helps reduce the rebuild time.

                       

                      It was just one disk failing intermittently out of a batch of 16 that I bought all at the same time and it was exchanged under warranty.

                       

                      If I also remember correctly, the initial formatting of this array took around 9 - 10 hours.

                      • 8. Re: RAID3 Rebuild Times
                        cfsKC Level 1

                         

                        I have now spoken with Areca tech support and was informed that:

                         

                        1.    For faster, he didn't know how much faster, rebuilds, I should disconnect the Areca battery backup unit I have installed on the controller.

                         

                        2.    He confirmed that enterprise drives will be faster on rebuilds and initializations than non-enterprise drives, such as my Samsung F3s, but he didn't know how much faster.

                         

                        3.    Increasing the cache on the controller card from 1GB to 2 or 4GB should also reduce the rebuild and initialization times but increase the boot up time.

                         

                        4.    He did not have a formula with which rebuild time can be predicted but, he said that he just had a 15TB RAID5 rebuild take 9 hours on a high performance, enterprise system.

                         

                        When I made the decision to go with an Areca controller and a RAID3 array, I was torn between it and the Intel controller and either RAID5 or RAID6 which Eric of ADK has extensively tested and steadfastly endorsed.

                         

                         

                        Had I been aware of the rebuild times involved with the Areca controller and the significantly faster rebuild times with the Intel card, the decision would have been much easier and I would not now be regretting my choice of the Areca controller and the added cost I incurred.

                        • 9. Re: RAID3 Rebuild Times
                          Harm Millaard Level 7

                          Had I been aware of the rebuild times involved with the Areca controller and the significantly faster rebuild times with the Intel card, the decision would have been much easier and I would not now be regretting my choice of the Areca controller and the added cost I incurred.

                           

                          Rebuild times with raid5 with Areca or Intel do not differ significantly. If anything, Areca will be faster than Intel because of the newer processor and the larger cache memory. With the number of disks in my array, a rebuild of a 12 disk raid50 would to my estimate take around 12 - 15 hours on the Intel and the Areca. Intel does not offer raid3 and the rebuild of raid3 is generally faster than raid5, due to the dedicated parity. How much faster is hard to tell without practical tests, but raid3 is definitely faster than raid5 to rebuild (unless the parity disk went down, in that case it is about as fast/slow as raid5).

                           

                          What can make a difference is the priority setting of the rebuild, low, medium or high. My disks are the slower F1's.

                           

                          Do not forget that Eric usually tests with far fewer disks, and both Scott and Eric find my array excessive, but if you hear from Areca that a 15 disk array takes 9 hours and you take my numbers, they seem to fit nicely.

                           

                          Rebuilding a raid5 requires reads from all drives, calculating parity and writing parity to all drives.

                          Rebuilding a raid3 requires reads from (n-1) drives, calculating parity and writing parity to a single disk. (lesser reads and lesser writes).

                          • 10. Re: RAID3 Rebuild Times
                            cfsKC Level 1

                             

                            Harm -

                             

                            Your insights and expertise are an essential component of this forum and have guided me and countless others to any number of good decisions in developing viable workstation for Premiere Pro.

                             

                            I make no claims to being anything but a near total boob when it comes to computers. My experiences with Areca controllers and RAID3, however, have been a disaster.  The initialization of a new Areca 1880ix took over 25 hours to complete. I have suffered through two rebuilds each lasting over 25 hours on a new systems with only eight 1TB disks. In the process I lost four days of production. That's 75 hours lost within one month on top of the extra cost of the Areca controllers.

                             

                            I have no experience with Intel controllers and make no claims about them or RAID5 or RAID6 from personal experience.

                             

                            Having had the very capable hands on assistance of Eric at ADK in this last rebuild of my RAID array, I can only report what I have learned from a professional who builds and tests video workstations every day.

                             

                            Had I had ADK build my system in the first place, I now know that I would not have experienced anywhere near the problems I have. I also know, that it would have cost me far less money. More importantly, I am quite confident that I would not have lost any production time.

                            • 11. Re: RAID3 Rebuild Times
                              Harm Millaard Level 7

                              No doubt about it, ADK and Eric in particular deliver great service. However, despite their professionalism, they can not guarantee there will be no disk failures at all. These things happen and if they happen, it is a PITA. In the normal situation formatting takes a long time but then runs without problems. However, even Eric can not prevent an occasional disk failure and if it happens it will cost time. If this occurs, you have basically two choices. Run the rebuilding at high priority and wait it out, or run the rebuilding at low priority and continue working, albeit at a slower pace. I guess the choice depends on the deadlines you have given your customers.

                               

                              Maybe Bill Gehrke can give you his recent formatting times with the Areca 1880iX-16. He has formatted a lot of different arrays in the last weeks.

                              • 12. Re: RAID3 Rebuild Times
                                ECBowen Most Valuable Participant

                                Well actually I am kind of puzzled by what Areca is reporting. The tech did not say how many disks were in the configuration. The tech just states 15TB. A 16TB array on a raid 6 even with the Intel only takes 5 hours and a raid 6 takes longer than a raid 5. Mind you that is 8 drives versus I don't know how many the tech reports. So right now we only have the verifiable results from his controller and those are really bad. A raid 10 should definitely not take over 20 hours for an 8 drive array. A raid 3 taking 25 hours is getting almost as bad as the onboard controller. So either the drives are the issue or the issue is the controller. If the controller is the issue then it's got to be their firmware or their driver. What you estimate is no where near what his results are. Since the Areca tech is obviously lost on this, we have to ask people who use the card. We need verifiable results.

                                 

                                BTW I set his priority at 80% or highest available. When it was not speeding things up, I set it to 50% since the Areca only gives the option for background tasks as a general setting that I could find. The Intel breaks the tasks down into each category, such as rebuild/ initialization and so on.

                                 

                                Eric

                                ADK

                                • 13. Re: RAID3 Rebuild Times
                                  Bill Gehrke Adobe Community Professional & MVP

                                  Incidentally, I have found a very good part of the initialization procedure for large arrays is to initialize first in RAID 0 where there is no initialization time.  Then run a write test before you mount it in Windows, I paid for and use HD Tune 4.60.  This is the way I weeded out a couple of non-array friendly commercial disk drives before going to the more complex and lengthy Initiialization time with the more complex and truely redundant RAID configurations.   I have not rebuilt an array but the initialization time on my Areca was 14.5 hours for a 12 each 1TB disk R30 array.

                                  • 14. Re: RAID3 Rebuild Times
                                    wonderspark Level 1

                                    I thought I'd share my experience here with RAID3 vs RAID6 on an Areca 1880ix-12 (with standard 1GB) using eight WD2003FYYS RE-4 disks and a Mac Pro.

                                     

                                    I first tried the RAID3 with six members+1 hot-spare. Initialization was about 35 hours. Never tested pulling a disk to rebuild because I knew it would take even longer, and I didn't have a weekend to spare.

                                    Next, I built a RAID3 with seven members+1 hot-spare. Initialization took 40 hours. Again, no time to test a rebuild.

                                    I used that setup for six months. My speed benchmarks using AJA in 1920x1080 10-bit mode, 16GB file (to overcome cache) gave sustained ~760MB/sec write, ~696MB/sec reads.RAID3 7HDD nocache.png

                                    I was happy with that, but always worried about how long a rebuild would take, and how it would run while degraded.

                                     

                                    Just this week, I had some free time and decided to test a RAID6 with the same setup.

                                    First, an 8-member RAID6 took only 4hrs 58min to initialize. I was thrilled!

                                    I tested the speed again:

                                    R6 16GB disab.png

                                    Not only a faster initialize, but faster performance. Great! So I pulled a drive to test speed while rebuilding, and got:

                                    R6 rebuilding 1 disk cache on.png

                                    R6 rebuilding 1 disk.png

                                    I tested with the cache on since that was what I'd be doing in the real world. I was impressed with the performance, and even went ahead and did some editing of full HD DSLR footage while it rebuilt.

                                    That rebuild took 7hrs 59min to complete, even with me working on a project.

                                     

                                    After these tests, with Areca's 1880 RAID card, I'm convinced that RAID6 is safer and better performing than RAID3 for my needs. Maybe it would have been more fair to build an 8-member RAID3 and have no hot-spare, but I wasn't comfortable with that plan six months ago. Anyway, just thought I'd share my results, given the crazy length of time it took to build a RAID3 with my system.

                                    • 15. Re: RAID3 Rebuild Times
                                      Scott Chichelli Level 3

                                      Eric and I have been saying exactly that from day 1

                                      thank you for taking the time to prove that

                                      Raid 3 bah humbug

                                      Raid 5,6 anyday

                                       

                                      Scott

                                      ADK

                                      • 16. Re: RAID3 Rebuild Times
                                        Harm Millaard Level 7

                                        Raises the question what is the reason to build an array?

                                         

                                        1. For performance, which means raid 3, or

                                         

                                        2. For rebuild times, which may mean raid 5/6.

                                         

                                        In my book reason 1 has preference over reason 2, but probably not in Scott's perspective. He would likely prefer slower performance, but faster rebuilds given his comment above. BTW, my personal experience with raid 3 rebuilds is that it is (much) faster than a raid 5 rebuild, due to the architecture of the array. Also take into consideration that Scott has limited experience with raid 3.

                                        • 17. Re: RAID3 Rebuild Times
                                          Scott Chichelli Level 3

                                          ummm i think you didnt read the above... Raid 6 was faster than 3 also what i have been saying...

                                          • 18. Re: RAID3 Rebuild Times
                                            Harm Millaard Level 7

                                            About the same with your raid results, that were way slower than mine, your own benchmark results that were more than 2 times slower than on my system, despite your i7-2600K, reputedly - at least by your standards about the fastest in the world -  versus my lowly 920. You can't seem to get your own benchmark results below 30 minutes, where I have no problem getting them in the 14 minute mark on a very slow CPU.

                                            • 19. Re: RAID3 Rebuild Times
                                              Scott Chichelli Level 3

                                              ummm and what does attempting to insult me have to do with the facts above or that you are wrong about raid 3

                                              keep it to the discussion at hand and lets not make it personal there Harm..

                                               

                                              that was a school girl answer

                                               

                                              Scott

                                              ADK

                                              • 20. Re: RAID3 Rebuild Times
                                                Harm Millaard Level 7

                                                From 'School girl' to Scott:

                                                 

                                                Read #7.

                                                • 21. Re: RAID3 Rebuild Times
                                                  wonderspark Level 1

                                                  For my needs, the RAID6 is a better match. I want to confidently edit full HD video, even if a drive fails, and I have proven to myself that I can do this in RAID6. If I ever get to a point where I can spare a week of time to re-test RAID3 with all eight members active (no hot spares), I will do that.

                                                   

                                                  I think there is something different with the 1880 series of Areca cards that makes them slower, but I hope that further testing can help narrow that answer down. For now, all I can say is I'm happier with my 8x2TB RAID6. I gave the RAID3 a try, but couldn't afford to fully test it right now due to my busy workload.

                                                   

                                                  To be continued... (dun-dun-dunnnnn)