1 2 3 4 Previous Next 141 Replies Latest reply: Mar 20, 2012 2:22 PM by Gf1317 Go to original post RSS
      • 80. Re: Storage rules for an editing rig. Some basics.
        kipp_litz

        Harm,

         

             Thank you for your "more than timely" reply.

         

             After researching your suggestions it appears that going the way of the SuperMicro CSE-M35T drive cage and a controller card might be my best option. If you do not mind I have a few questions.

         

             If I go with an Areca ARC-1231ML card, 2 of the SuperMicro cages per machine, and 6 x 2 TB drives, will I be able to set up two independent RAID 5 arrays (3 drives each array) on the the card? This would be obviously one array for media/CS project files and the other for PF/Scaratch/Render. I have used RAID cards before but nothing in the league of the Areca is why I am asking.

         

             Assuming same drive cages, same Areca card and drives moved in to exact same positions (same SATA port as well), moving form machine A to machine B would be as easy as pulling all 6 drives and re-inserting them?

         

             Finally, if I also want external SATA access (non RAID) connectivity am I going to be forced to add another card or does Areca make a card that will fill both duties? I am looking around their site and nothing is jumping out at me. I guess I could simply add a e-SATA plate to the back of the machine and cable it to a few free SATA ports on the motherboard.

         

             And yes, the OS (Win 7 64 bit) and programs would reside on a WD Raptor.

         

             Thank you for your valuable time and insight.

         

        Kipp Litz

        • 81. Re: Storage rules for an editing rig. Some basics.
          Harm Millaard CommunityMVP

          Better get the Areca ARC-1680iX-12 . It has a better and faster IOP, allows for 12 disks internally and 4 externally.

           

          You can create multiple raid arrays and pass-thru disks (single disks).

           

          Here is how I set up my array:

           

          30-4-2010 8-48-03.jpg

           

          So with two different raid3 arrays, that can be left as such, but I chose to stripe them in a raid30, one big volume.

          • 82. Re: Storage rules for an editing rig. Some basics.
            FelixUnderwood Community Member

            So Harm, I have a 256Gb Corsair SSD (C Drive, of course), and 2 x 2Tb Seagate Sata III HDs in a RAID 0 for Media & Scatch (Preview Files in PPro). Should I be using one of the two 2Tb drives for Page & Scratch and the other strictly for media??

            • 84. Re: Storage rules for an editing rig. Some basics.
              klfi Community Member

              Thanks Harm, again great articel.

               

              In the last mounth i followed all your articles and "rules" and have now a much faster running system.

              One thing is not clear for me

              Rule 3: 7.200 or 10.000 RPM and more single discs if possible

               

              Im using at the moment

              Drive D: pagefile, render, prev, ...

              3x WD each 300 GB 10.000 RPM raid 0 Areca controller = 900 GB 10.000 RPM raid 0

               

              to compare, would it be better to use 1 or 2 Spinpoint F1 each 1 GB 7.200 raid 0 Areca controller = 2 GB 7.200 RPM

              (BTW, variant 2 seems to be cheaper)

               

              In which relation are storage space (raid 0 configered) and RPM staying ?

               

              klfi

              from Austria   

              • 85. Re: Storage rules for an editing rig. Some basics.
                Harm Millaard CommunityMVP

                Klaus,

                 

                I'm not sure  understand your question correctly, but here are my thoughts:

                 

                A single Velociraptor will achieve an average sustained transfer rate of around 107 MB/s with an average access time of 7.0 ms

                 

                HD Tach C.jpg

                 

                A single Spinpoint F1 will show around 99 MB/s and access time of 13.6 ms

                 

                HD Tach B.jpg

                 

                A two disk Spinpoint F1 array will show figures like this:

                 

                HD Tach D1.jpg

                 

                What you can derive from this, is that at least for raid0, there is hardly any overhead. Average speed of the raid0 is directly porportional to the number of disks in the array, so with 3 disks in a raid0, it will be about 3 times faster than a single disk, with 4 disks, 4 times as fast.

                 

                However, since raid0 has no redundancy whatsoever, the chance of losing all your data is also directly proportional to the number of disks in the array. Read the Raid article, that I linked to in a previous post, for more info.

                 

                In your case you can expect your current raid with 3 Velociraptors to achieve slightly less than 300 MB/s transfer rate versus around 180 MB/s for two Spinpoints.

                 

                RPM is important for access time.

                 

                If this did not answer your question, let me know.

                • 86. Re: Storage rules for an editing rig. Some basics.
                  klfi Community Member

                  Thanks Harm for yr answer

                   

                  i think i understood 60%, my interpretation says 3x VelRap (900 GB) raid0 are  much faster than 2x Spinpoint (2 TB) raid0

                   

                  let me summarice:

                  1. raid0 is directly porportional ... i understand

                  2. redundancy.... i understand

                   

                  and further about acsess time:

                  3. 3x VelRaptor (each 300 GB) Raid0 = 900 GB =  data transfer 300 MB/s = not understand acess time 7 msec or 3x 7 msec ??

                  4. 2x Spinpoint (each 1 TB) Raid0= 2 TB = data transfer 180 MB/s = not understand acess time 13,6 msec or 2x 13,6 msec ??

                   

                  let me ask,

                  5. whats more important, what priority we should take care of: data transfer rate or acess time

                    or does the general storage rules say "as less acess time as more data transfer" - or vice versa. What are the relations?

                  6. my blackmagic dicspeed tool shows me "read" and "write" transfer rate, but no acess time. what is more important for editing, read or write?  

                   

                  Thanks, Klaus

                  • 87. Re: Storage rules for an editing rig. Some basics.
                    tclark513 Community Member

                    When turning the indexing off do you want to do c:\ only or also the folder and subfoder option as well?

                    • 88. Re: Storage rules for an editing rig. Some basics.
                      Harm Millaard CommunityMVP

                      Yes, I do. on the whole disk, including directories and subdirectories, apart from the ones that Windows objects to.

                      • 89. Re: Storage rules for an editing rig. Some basics.
                        tclark513 Community Member

                        Thanks harm...

                         

                        What size paging file do you recommend if you have 12gig of ram?

                        • 90. Re: Storage rules for an editing rig. Some basics.
                          tclark513 Community Member

                          Anybody care to comment on what their findings were when changing their pagefile and what size works best for 12gigs of ram.  Thanks!

                          • 91. Re: Storage rules for an editing rig. Some basics.
                            Harm Millaard CommunityMVP

                            I have 8 GB fixed on a 2 disk raid0 with 12 GB memory and it is not used often. It suffices for my workflow. If you are heavy into DL'ing to AE and EN, often have PS and AU open, you may need to set it to 12 GB.

                            • 92. Re: Storage rules for an editing rig. Some basics.
                              Bill Lake Community Member

                              (Perhaps I should start a new thread)

                               

                              I want to optimize storage for video editing on a PC In Premiere Pro CS5, and I have not seen the information that I'm looking for after reading a lot of articles about sustained data rate being key, but, given limited budget and drives, I'm not clear where it is needed most. I am forced to reconfigure my system for unrelated reasons, so I want to get it right. My goal is to be able to avoid framedrops, hangs, or long render times. My source files are always provided on a portable firewire 7200 drive. I typically edit long files down to  20 minutes or less, with motion graphics, multiple video tracks and effects, etc.

                               

                              So, can someone educate me:

                               

                              - To minimize bottlenecks, I understand  reading and writing should be done on different disks, but I'm not clear on where the priority goes. I'm not sure what is being read and written when rendering takes place (assuming that's when it matters.) Should capture, rendering and scratching be done on 3 different disks. And if so, what's the priority for performance. When rendering is PP reading from scratch and the captured disks or just scratch at that point? Can anyone help me understand a little more here- what files are being accessed when rendering for export?

                               

                              - Where do I store graphics and sound effects that I want to use in the project(s)?

                               

                              - I have a limited number of drives - where does sustained throughput matter most? Scratch? Rendered? Source? Can you set priorities?

                               

                              - Also, if possible what should NOT be put on the same drive?

                               

                               

                              Or better yet - tell me what I should do: I have a quadcore PC w 8G ram,  2x8800gtx w768M. I have 4 internal drives:  Raid0 with 2x160G drives 7200 rpm, and  I was thinking I would put my OS and apps on that. Then I have 2 internal 7200rpm 1.5G drives and I was thinking of making one for scratch and images and the other for render (both video and audio). I can use external USBs for other non-editing documents. Is this a good solution? Alternatively - if sustained throughput is more important than separating read/write, I could make a Raid0 from the 2x1.5G drives. Should I buy an esata card to connect my source media that way? (not sure I can fit the card)

                               

                               

                              Thanks

                               

                              -Bill

                              • 93. Re: Storage rules for an editing rig. Some basics.
                                Harm Millaard CommunityMVP

                                (Perhaps I should start a new thread)

                                 

                                That makes sense. Better chance on getting valuable reponses than in this old thread.

                                • 94. Re: Storage rules for an editing rig. Some basics.
                                  TKlopp

                                  Hi Harm,

                                   

                                  Your info is very helpful - and I understand 3 physical drives is a best case scenario. I have two questions.

                                   

                                  I am trying to run CS5.5 on a laptop with this set up:

                                  Dell Latitude 6520

                                  Intel Cor i7 2720QM CPU @ 2.20 GHz

                                  8.00 GB RAM (7.88 GB usable)

                                  Win 7 Ultimate 64-bit

                                   

                                  I currently have 2 internal drives in the machine, and both are formatted NTFS.

                                   

                                  The main drive is 450 GB and the second is 300 GB... Again realizing 3 HD is recommended, if you had to work with two - how would you align project files, raw files, media cache, and scratch disks.

                                   

                                  Also - I may be able to get an external 1TB HD that hooks up via ATA (not USB!). If I add this to the mix, how would you suggest I alter the file set up?

                                   

                                  Thanks so much!

                                   

                                  Ted

                                  • 96. Re: Storage rules for an editing rig. Some basics.
                                    TKlopp Community Member

                                    Awesome article - thanks so much!

                                     

                                    Ted

                                    • 97. Re: Storage rules for an editing rig. Some basics.
                                      darrillandries

                                      Wow, a lot of info.  I have a JVC camera that creates MTS files. I want to buy adobe premier pro and have a computer built(PC tower)  specifically for making movies. I am an amateur for sure and do this as a hobby. What kind of of optimal system requirements should I ask for to make it run fast. I hate the skipping and waiting while your machine lags behind your mind. Any advice would be welcome. Thank you 

                                      • 98. Re: Storage rules for an editing rig. Some basics.
                                        Harm Millaard CommunityMVP

                                        Start with the overview tab at the top of the hardware forum and start reading.

                                        • 99. Re: Storage rules for an editing rig. Some basics.
                                          Frederic Segard Community Member

                                          removed

                                           

                                          Message was edited by: Frederic Segard

                                          • 100. Re: Storage rules for an editing rig. Some basics.
                                            dec0n

                                            Hello guys,

                                             

                                            Another FCP switcher here  ^_^

                                             

                                            I'm in the middle of building a rig, and have opted for a 6 disk areca controlled RAID3 setup similar to many threads on this forum.

                                             

                                            I am now wondering about which drives to put in there, I know the Samsungs are popular, and of course no green disks etc. 
                                            Would it a big enough performance increase to put 6 velociraptors in there for the price?
                                            Thanks for any help or advice! 
                                            ^_^

                                            • 101. Re: Storage rules for an editing rig. Some basics.
                                              Frederic Segard Community Member

                                              Welcome fellow switcher!

                                               

                                              A good drive, tested by Areca with their controlers, very affordable, and a good all around performer, is the Hitachi 7K300 series. Raptor drives in a RAID is kind of useless. Way to expensive for very little gain (at that price, might as well take SSDs). But at roughly $65 for a 1.5TB SATA III drive, you can install drives to your heart's content (or to the maximum number of ports on your card, which ever comes first), and see a tremendous gain in speed.

                                              • 102. Re: Storage rules for an editing rig. Some basics.
                                                RFDPiper Community Member

                                                 

                                                Rule 5: Turn off index search and compression. Both will cause severe performance hits if you leave them on.

                                                 

                                                 

                                                Sorry, I have confusion about this.  What does this refer to and where is this done?  In the OS or in Adobe?

                                                • 103. Re: Storage rules for an editing rig. Some basics.
                                                  RjL190365 Community Member

                                                  RFDPiper wrote:

                                                   

                                                   

                                                  Rule 5: Turn off index search and compression. Both will cause severe performance hits if you leave them on.

                                                   

                                                   

                                                  Sorry, I have confusion about this.  What does this refer to and where is this done?  In the OS or in Adobe?

                                                  In the OS.

                                                  • 104. Re: Storage rules for an editing rig. Some basics.
                                                    RFDPiper Community Member

                                                    Thanks for the fast reply, but where in the OS?

                                                    • 105. Re: Storage rules for an editing rig. Some basics.
                                                      RjL190365 Community Member

                                                      Go to "My Computer", right-click on the disk that you want to change the settings on, select "Properties", then uncheck both of the boxes at the bottom of the "General" tab (labeled "Compress this drive to save disk space" and "Allow files on this drive to have contents indexed in addition to file properties"; by default, the latter check box is checked).

                                                      • 106. Re: Storage rules for an editing rig. Some basics.
                                                        RFDPiper Community Member

                                                        Thanks RjL,

                                                        Two more questions: 

                                                         

                                                        1) I have separate drives for media, OS/Programs, and render.  I am setting up my RAID 0 render drive right now (don't worry, I regularly back-up to a RAID 1 NAS), and in order for the computer to see my drive obviously I have to initialize it.  It's asking me if I want to initialize it as a MBR or a GPT.  Not sure which to choose, please help.  I'm assuming MBR is OK since I'm well under 2TB. 

                                                         

                                                        The drive is two 450GB Velociraptors RAIDed off of the motherboard controller if that makes a difference (I know, I know, get a hardware controller right?  Well, someday).

                                                         

                                                         

                                                        2)

                                                        Rule 1: NEVER partition a disk.

                                                         

                                                        I set my C: drive (OS/programs) to MBR and it seems to have partitioned itself.  Should get rid of the partition, or does that not make a difference on the C; drive?  I don't mind reinstalling my OS and everything to get the best setup

                                                        Capture.PNG

                                                        I'm using Windows 7 Professional 64-bit

                                                        • 107. Re: Storage rules for an editing rig. Some basics.
                                                          RFDPiper Community Member

                                                          RjL190365 wrote:

                                                           

                                                          Go to "My Computer", right-click on the disk that you want to change the settings on, select "Properties", then uncheck both of the boxes at the bottom of the "General" tab (labeled "Compress this drive to save disk space" and "Allow files on this drive to have contents indexed in addition to file properties"; by default, the latter check box is checked).

                                                          I get a promt when I uncheck the "Allow files on this drive to have contents indexed in addition to file properties" box.  Should I "Apply changes to drive C:\ only"  or  "Apply changes to drive C:\, subfolders and files"?

                                                          • 108. Re: Storage rules for an editing rig. Some basics.
                                                            JEShort01 Community Member

                                                            Use the later, "Apply changes to drive C:\, subfolders and files".

                                                             

                                                            Jim

                                                            • 109. Re: Storage rules for an editing rig. Some basics.
                                                              tomglen Community Member

                                                              Hi Harm,

                                                              Was wondering if you could advise. I was looking at Rule #3 on your guide. I'm not sure I can afford 3 discs so was considering a 2 Disc set up (2x 1TB Seagate Barracuda 7200rpm, 32MB Cache). I'm mainly using CS4 Ppro for HD video (<15mins in length) and some after effects. How crucial is a 3 disc set up? Will I still get good speed on that set up? And is it worth upgrading to 64MB caches?

                                                               

                                                              spec is:

                                                              Intel i7 2600K 3.4GHz

                                                              Asus P8Z68-V Pro

                                                              8GB RAM

                                                              Nvidia EVGA GTX 550Ti

                                                              2x 1TB Seagate Barracuda 7200rpm, 32MB Cache

                                                              Windows 7 Pro

                                                               

                                                              Thanks a lot

                                                              Tom

                                                              • 110. Re: Storage rules for an editing rig. Some basics.
                                                                Harm Millaard CommunityMVP

                                                                Given that you already have two disks, the investment would be one additional disk. You would definitely notice that in performance. Changing disks to 64 MB cache models would only marginally improve performance if at all and would be much more costly.

                                                                 

                                                                The half duplex nature of SATA disks means that traffic in one direction (reading or writing) must finish first before traffic in the other direction can begin. That causes delays you would not have with another disk, because one can be used for traffic in one direction and the other disk for traffic in the other direction.

                                                                • 111. Re: Storage rules for an editing rig. Some basics.
                                                                  tomglen Community Member

                                                                  This is for a new system and I have limited budget so trying to work out what's best to spend my money on. How much will this affect workflow during projects or is it just the opening and saving of projects? Also, would it realistic to add a third external SATA drive at a later date if I need it?

                                                                   

                                                                  Thanks for the help,

                                                                  • 112. Re: Storage rules for an editing rig. Some basics.
                                                                    Harm Millaard CommunityMVP

                                                                    Get a third disk for OS & programs, like the Samsung Spinpoint 320 GB F4 at € 70. Makes better use of your intended 2 x 1 TB disks as well, giving you more storage space and dividing the work across more disks.

                                                                    • 113. Re: Storage rules for an editing rig. Some basics.
                                                                      tomglen Community Member

                                                                      What do you reckon to an SSD for OS & Programs drive? Would you recommend? Was looking at 128GB Corsair Performance Pro

                                                                      • 114. Re: Storage rules for an editing rig. Some basics.
                                                                        Harm Millaard CommunityMVP

                                                                        SSD is in conflict with a tight budget.

                                                                        • 115. Re: Storage rules for an editing rig. Some basics.
                                                                          tomglen Community Member

                                                                          yeah you have a point there. Thanks so much for the advice Harm, it's invaluable having your knowledge on tap!

                                                                           

                                                                          Okay, here's the setup i'm now looking at and the uses for each drive. Do you think this will work well?. I know it uses a green drive which you weren't a fan of but the company we're ordering through only offer those for a third drive. (unless you get an SSD for drive one) and i'm guessing this will still be superior to a 2 drive setup?

                                                                           

                                                                          Drive 1: 1TB Seagate Barracuda 7200rpm 32MB Cache.

                                                                          Drive 2: 1TB Seagate Barracuda 7200rpm 32MB Cache

                                                                          Drive 3: 1TB Western Digital Caviar Green, 64MB Cache

                                                                           

                                                                          Drive 1: OS, Programs, Pagefile

                                                                          Drive 2: Media, Projects

                                                                          Drive 3: Previews, Media Cache, Exports

                                                                           

                                                                          My other option is to go for a SSD as drive one but to keep it on budget I think that would have to mean downgrading the motherboard from a Asus P8Z68-V PRO  to a  Gigabyte GA-Z68XP-UD3 and I don't have a clue how much difference that would make to performance. Any help would be much appreciated!

                                                                          • 116. Re: Storage rules for an editing rig. Some basics.
                                                                            tomglen Community Member

                                                                            Okay, here's the setup i'm now looking at and the uses for each drive. Do you think this will work well?. I know it uses a green drive which you weren't a fan of but the company we're ordering through only offer those for a third drive. (unless you get an SSD for drive one) and i'm guessing this will still be superior to a 2 drive setup?

                                                                             

                                                                            Drive 1: 1TB Seagate Barracuda 7200rpm 32MB Cache.

                                                                            Drive 2: 1TB Seagate Barracuda 7200rpm 32MB Cache

                                                                            Drive 3: 1TB Western Digital Caviar Green, 64MB Cache

                                                                             

                                                                            Drive 1: OS, Programs, Pagefile

                                                                            Drive 2: Media, Projects

                                                                            Drive 3: Previews, Media Cache, Exports

                                                                             

                                                                            My other option is to go for a SSD as drive one but to keep it on budget I think that would have to mean downgrading the motherboard from a Asus P8Z68-V PRO  to a  Gigabyte GA-Z68XP-UD3 and I don't have a clue how much difference that would make to performance. Any help would be much appreciated!

                                                                            • 117. Re: Storage rules for an editing rig. Some basics.
                                                                              RjL190365 Community Member

                                                                              In this case, get another 7200 RPM hard drive from another source: Green drives are often too slow in both sequential transfer speed and random access speed to be of much use for anything besides backups. In the case of the Caviar Green, its rotational speed is only 5400 RPM despite Western Digital's "variable-speed" claim.

                                                                              • 118. Re: Storage rules for an editing rig. Some basics.
                                                                                tomglen Community Member

                                                                                Thanks for that. Would an external eSATA connected drive be fast enough do you think?

                                                                                • 119. Re: Storage rules for an editing rig. Some basics.
                                                                                  RjL190365 Community Member

                                                                                  In this case, get an external USB 3.0 or eSATA enclosure and put the Green drive into that. Then get the third internal 7200 RPM hard drive.