12 Replies Latest reply on Nov 1, 2010 12:58 PM by Scott Chichelli

    Please help me adjust my current HDD configuration

    davidbeisner2010 Level 3

      I posted this at the end of the "Guidelines for Best Disk Setup" by Harm yesterday, but I'm thinking that thread may be so long and so old that my question simply got lost in the mix, so I'll go ahead and create a new post here.

       

       

      I've read Harm's post on guidelines for best disk setup and his post on general storage setup for editing systems, and I've got some questions...

       

      First, I've got a TON of data, much of which I could probably archive to clean things up. Here's what I've got right now, and maybe you can give me some recommendations for fixing things so it'll run better (my PPBM5 shows significant slowdown in disc-land). FWIW, all internal drives are 7200RPM SATA2.

       

      I have a 250GB WD Enterprise Storage drive for my OS/Programs, and I think it holds the pagefile... (though I have 12GB of Ram, so I'm not sure how often it's used). This one is about 50% full.

       

      I have a 500GB WD Enterprise Storage drive which holds all my graphics, photos, and audio, along with some other generic data files (think documents, PDFs, etc. here) This one is about 60% full.

       

      I have a 1.0 TB WD Caviar Black for my Projects/Scratch/Previews disc. This one is about 40% full.

       

      I have a 2TB RAID-0 (made from 2x 1.0TB WD Enterprise Storage drives) that holds all my data in whatever format it came in on (most of it is direct capture from HDV in Premiere, but there's a good mix of AVCHD, R3D, and various SD stuff, too). This one is about 95% full.

       

      I also have a 4TB RAID-0 WD backup drive that's a little over 50% full, and maintains a complete backup of all the drives in my system, minus OS and Programs. (FW800)

       

      Then I have a second 2TB single disc drive that I use to maintain an off-site backup of whatever projects I'm currently working on, plus important stuff like photos and graphics. This one is about 30% full. (FW800)

       

      Then I have a third 1TB single disc drive that is a general backup drive and usually remains onsite, and is about 80% full. (USB 2.0).

       

       

      Ok, so there's where things are right now. Based on what I've read, I'm thinking that I might be better off doing the following:

       

      250GB for OS/Programs

      500GB for Previews/Scratch/Pagefile

      3x 1.0TB drives (no RAID) for my projects/media OR 1.0TB in RAID3/5 for projects/media (but I'll have to figure a way to archive a bunch of stuff... Bluray?)

       

      I've got room in my system to add one more drive, and I can probably get approval for purchase of one more drive if I asked my boss--so if I got a 2TB drive, could I do a 2.0TB RAID using two of the 1.0TB drives for striping and the 2TB for parity, leaving me with an extra 1.0TB? Or should I get a 1TB drive and do a 2.0TB Raid with two drives for striping and two for parity? I'm really not up on this RAID stuff, despite reading your article on it and Wiki...

       

      Also, could I separate my current RAID-0 and not lose the data on it?

       

      And just FYI, I'm using onboard RAID (using the ASUS P6T6 Server mobo), and purchase of a RAID controller is out of the question right now...

       

      I primarily edit HDV/AVCHD, but I have a little bit of RED 4k and I'm hoping to upgrade my HDV camera to the JVC GY-HM700 soon.

       

      Thanks for your advice and expertise!

       

      David

       

       

       

       

       

        • 1. Re: Please help me adjust my current HDD configuration
          Scott Chichelli Level 3

          OS drive (and pagefile)

           

          set of 2 or more raid 0 or 5 scratch/capture/work what ever you want to call this where you ingest your footage and work from

          set of 2 or more raid 0 or 5 for output/render to.

           

          external back up NAS/eSata etc

          no need for anything more complicated than that.

           

          Scott

          ADK

          1 person found this helpful
          • 2. Re: Please help me adjust my current HDD configuration
            Harm Millaard Level 7

            David,

             

            First a couple of general remarks.

             

            Without a raid controller and only having the mobo raid capabilities, it hardly makes sense to use anything but raid0,

             

            All disks in a raid array must be identical for an effective raid, otherwise you are talking about a JBOD (Just a Bunch of Disks) and the mobo does not support that, let alone the performance penalty.

             

            For more background on raids, read here: Adobe Forums: To RAID or not to RAID, that is the...

             

            You documented your question well, but I get the feeling that your backup strategy is not very clear, you are using multiple externals for partial backups and possibly some duplication of data. It sounds a bit disorganized.

             

            Your idea about the 250/500/3x1TB makes sense, but what about all the other drives you are using? Can you give a clear listing of your drives and interfaces. Remember, for raids the disks need the same capacity and the same SATA interface. FW800 or USB2 can not be raided on the mobo.

            • 3. Re: Please help me adjust my current HDD configuration
              davidbeisner2010 Level 3

              Thanks Harm... sorry, that really wasn't as clear as it could have been. Here's what I've got right now:

               

              (ALL DISCS ARE SATA2 7200)

               

              ONE 250gb OS/Programs/Pagefile

              ONE 500gb for photos/graphics/audio-only projects using Audacity/other documents

              ONE 1tb for Projects and Outputs

              TWO 1tb as RAID-0 for video data (mix of direct-ingest of AVCHD and R3D data files and captured HDV from Premiere).

               

               

              For externals, I have ONE 4tb FW800 that maintains a running backup of all drives EXCEPT OS/Programs and ONE 2tb drive that I use to backup important graphics/photos and also both video data and project files of active projects, which is generally kept off-site (in case of fire or a massive electrical surge that blows out my UPS and computer...)

               

              After reading your info, it looks like I might be better off re-doing my discs such that my projects and video data are on the same drive and the outputs are on the separate drive (instead of separating projects and data). Is that the case? And if so, what would be the best way to do that with what I have? And is RAID-0 even necessary for the sort of stuff I edit? Do I get enough speed increase from it to make things better, or would I be better off not doing RAID at all?

              • 4. Re: Please help me adjust my current HDD configuration
                Harm Millaard Level 7

                Thank you, David, that makes it a lot clearer. At least much more understandable for my tired mind now.

                 

                Given your options (no separate raid controller, possibly one or even two extra disks), I would go for this setup:

                 

                C: 250 G OS & programs

                D: 500 G photos/graphics/audio-only projects using Audacity/other documents & Pagefile

                E: 1 TB Projects and Outputs & Media Cache

                F: 3 x 1 TB Raid0 for media and previews

                 

                The reasoning behind this is that currently you are running out of space on your 2 disk raid0, during editing your access to current contents on D will be rather small, so relocating your pagefile may be attractive and by using the E drive for the media cache, next to projects you are splitting the load between E and F. Your F drive grows in capacity and will be quicker and the backups are taken care of with both the 4 TB and the 2 TB off-site disks.

                • 5. Re: Please help me adjust my current HDD configuration
                  davidbeisner2010 Level 3

                  Thanks, Harm... I'll put in the request for a new drive on Monday, and see what happens! Short of a new drive, I'll just keep it as is (except moving the pagefile) and move as much as I can from the data drive to archive--is bluray the best way to do that at this point? I'm leery of archiving hard drives as I've read the platters can stick if they're not spun up frequently...

                  • 6. Re: Please help me adjust my current HDD configuration
                    Harm Millaard Level 7

                    The most practical solution would be IMO to add a 2 TB disk to your system as well, not only a 1 TB disk as a third disk on your raid0. If you do that you can simply copy the contents of your current raid to this new 2 TB disk, expand your raid, move the data back to this new raid0 and then allocate this new G drive for media cache, previews and exports. It will save you around 50 BRD's and save you lots of time. Just the cost of 50 BRD's will compensate the cost of a 2 TB disk and that is not calculating your time.

                    • 7. Re: Please help me adjust my current HDD configuration
                      davidbeisner2010 Level 3

                      Hmm... Good idea, but I've only got one SATA plug left available on my mobo, so short of also getting a dedicated RAID card (and I've read the only ones worthwhile cost close to $1k and up, so that's out of the question), I can only add one more drive, not two.

                      • 8. Re: Please help me adjust my current HDD configuration
                        Harm Millaard Level 7

                        You may be able to use an eSATA connector, temporarily disconnect a SATA DVD/BR burner just for the data transfer or get a PCIe-SATA card. Just use your imagination, but I think that you can save lots of time by figuring out a solution to the lack of SATA ports on your mobo.

                        • 9. Re: Please help me adjust my current HDD configuration
                          davidbeisner2010 Level 3

                          Duh... I should have thought of those things! Unfortunately my onboard eSATA hasn't worked since day one, despite nearly two hours with a BOXX technician remoting into my computer working on it... but a PCI card would be sweet! I'll look into that!

                          • 10. Re: Please help me adjust my current HDD configuration
                            Scott Chichelli Level 3

                            why  complicate the drive configuration?

                             

                            what i recommended only requires 5 Sata ports. however the separate Audio/photos makes 6

                            moving the pagefile off the OS had absolutely no effect in performance (unless the OS is a small SSD)

                            we benchmarked this by moving the page file from the OS to standard Sata, Raptor and SSD.

                            0 performance gain. And thats all the drive was used for.

                             

                            AVCHD and Red should both have Raid 0 drives.

                            FYI an excellent raid card is only $500 Intel RS2BL080

                             

                            your output drive should be the same speed as the media drive.

                            you have 4 drives for those 2 items. better to have both of them raid 0

                            2 each. that way your renders do not take as long. (also more space)

                             

                            and Harms idea of eSata ext. is a good one. i prefer a raid 5 NAS but they can get pricy

                            just in case you are not aware your drives should not be filled past 65% MAX. you get a serious performance hit.

                             

                            Scott

                            ADK

                            • 11. Re: Please help me adjust my current HDD configuration
                              davidbeisner2010 Level 3

                              Wow... just moved my pagefile and I can already see a speed increase just in the responsiveness of programs and stuff within Windows! I've put in the request for a third 1TB drive for the array, and I'm hoping to get it... I'll post back results on what happens if I'm able to get it and add it to the system.

                              • 12. Re: Please help me adjust my current HDD configuration
                                Scott Chichelli Level 3

                                only way thats possible is if your OS drive was full...