3 Replies Latest reply on Jan 19, 2014 11:46 AM by JEShort01

    Disk configuration and workflow help needed for lab video workstation

    Xetred

      Hi All,

       

      Setting up a video editing workstation for a research lab that will use Premeire to edit AVCHD Progressive clips (sometimes with 2 streams side-by-side, but usually single-camera) and export them to .mp4 for later viewing by video coders. We won't be using AfterEffects or adding anything to the videos other than some text (titles, maybe sub-titles).

       

      The other purpose of this workstation is to act as a file server and backup system for other machines in the lab. Coders will be viewing the exported videos via other networked machines and working with Microsoft Office files that will be stored on the workstation's other HDDs. I'll have a physical backup drive and cloud backup via CrashPlan.

       

      I've built a machine that is probably overkill, but the client (my wife) wanted it to be "fast," and the purpose of the machine might change in the future:

       

      i7-4770K (overclocked a bit)

      16GB RAM

      Asus Z87-Pro

      GeForce GTX 660

       

      I have the OS (W7) and programs on a 256 GB Samsung 840 Pro SSD and currently have two 1TB Velociraptors to use for the Premiere workflow. I'm trying to figure out how to proceed with the purchase of the rest of the drives, and I want to keep the Premiere drives separate from the large storage drives from the lab that are networked and synced to cloud backup.

       

      Following the recommendations for a three-disc configuration I've picked up on these forums, I could set it up like this:

       

      C: (256GB SSD) (OS, programs, pagefile)

      D: (1TB HDD) (media, projects)

      E: (1TB HDD) (previews, media cache, exports)

      ________________________________________

       

      F: (4TB HDD) (backups of media, projects, and exports and storage of other research files)*THIS DRIVE WOULD BE SHARED ON THE NETWORK

      G: (4TB external HDD) (backup of F & drive that backs up to CrashPlan)

       

      but it seems that would be a waste of the speed of the second 10k velociraptor. If I added another SSD and RAIDed the Velociraptors it would be:

       

      C: (256GB SSD) (OS, programs)

      D: (Two 1TB Velociraptors in RAID 0) (media, projects)

      E: (256GB SSD) (media cache, pagefile)

       

      but would I then need to add another dedicated HDD for previews and exports, or could I store those on the networked F: from above (which would be previews, exports, backups of media and projects, and storage of other research files) without taking a speed hit?

       

      It seems overkill to have a dedicated drive for exports and previews (let's make that the new F:), then have them copy to the first 4TB drive (now G:), then back that up to the second 4TB drive (now H:), then back that up to CrashPlan. However, people might be accessing that network drive at any time, and I don't want that to slow any part of the video process down.

       

      I appreciate any advice ya'll can give me!

        • 1. Re: Disk configuration and workflow help needed for lab video workstation
          JEShort01 Level 4

          Kudos to you for your very well thought out post!

           

          I would actually suggest that you assemble and start using what you have planned without the added SSD and test it out:

          - Is timeline work smooth and with an acceptable number of dropped frames for playback without rendering?

          - Are render outputs fast enough to be "acceptable" to you and your wife?

           

          Then, if you need more speed, you will have some testing under your belt to access how your new build is leaving you short.

           

          On another note, if I were you I would start with the two Velociraptors in a RAID 0 for all files from the start. You can also configure and compare the two-drive configuration vs. a RAID 0 configuration very easily since you have some excellent attached backup storage already.

           

          If you want to get more scientific still, the PPBM6 Premiere Pro benchmark is excellent to see how well your build is performing for various tasks and is also an excellent metric to use when trying out different drive configurations like the two-drive vs. RAID 0 question.

           

          It is my person belief that RAID 0 is better today even though separate drive setups were preferred a few years ago (Harm's guides, Adobe 2012 white paper, etc.). Windows 7 (and 8, and 8.1) as an operating system seems to be using now common RAM sizes of 16GB and up to be utilized very effectively for drive caching.

           

          Regards,

           

          Jim

          • 2. Re: Disk configuration and workflow help needed for lab video workstation
            Xetred Level 1

            Hi Jim,

             

            Thanks for the encouraging response. I'm leaning toward the non-SSD option at this point. 

             

            To make sure I understand, are you suggesting I try using the Velociraptor Raid 0 in the 2 disk configuration suggested by Harm's Guidelines for Disk Usage chart? Like this:

             

            C: (256 GB SSD) (OS, Programs, Pagefile, Media Cache)

            D: (1TB x2 in RAID 0) (Media, Projects, Previews, Exports)?

             

            Where I'm still confused there, and in looking at Harm's array suggestions for 5 or more drives, is how performance is affected by having simultaneous read/write operations happening on the same drive, which is what I understood was the reason for spreading out the files on multiple drives. Maybe I don't understand how Premiere's file operations work in practice, or maybe I don't understand RAID 0 well enough.

             

            In the type of editing we'll be doing (minimal) aren't there still times when Premiere will be trying to read and write from the D: drive at the same time, for example during export? Wouldn't the increased speed benefits of RAID 0 for either read or write alone be defeated by asking the array to do both simultaneously?

             

            Maybe the reason the Media Cache is on the SSD in the above configuration is because that is what will be read while writing to something like Exports? But that wouldn't make sense given Harm's chart, which has the Media Cache also located on the array....

             

            Another question is, given that the final home of the exported videos will be on the big internal drive (4TB) anyway, could I set it up like this:

             

            C: (SSD) (OS, Programs, Pagefile, Media Cache)

            D: (2TB RAID 0) (Media, Projects, Previews)

            E: (network shared 4TB HDD) (Exports + a bunch of other shared non-video files)

             

            so I don't end up having to copy the exported videos over to the 4TB drive? Do you think it would render significantly faster to the RAID than it would to the 7200 rpm 4TB drive? I'd like to cut out the step of copying exported videos from D: to E: all the time if it wasn't necessary.

             

            Thanks again.

            • 3. Re: Disk configuration and workflow help needed for lab video workstation
              JEShort01 Level 4

              First, it seems an explanation regarding RAID 0 might help...

               

              With RAID 0, reads or writes are darn near twice the throughput as with a single drive. RAID 0 is "striping" or whenever you do a write you are writing half of the data to one drive and half of the data to the other. Then on reads, you see the same almost doubling as well. It is not a complete doubling of speed due to some additional overhead.

               

              Now, what can caching do for us? Drives and also the Windows operating system provide caching that is quite effective at smoothing over the various simultaneous reads and writes to our drives. Back a few years ago when a PC had 8GB or 12GB or RAM and drives had up to 32MB of cache on the drives themselves. Today, it is not uncommon to have PCs with 16, 32, or even 64GB or RAM installed and the cache size on individual drives can be up to 128MB. With Win7 drive policies set to "write-caching" enabled and "windows write-cache buffer flushing on the device" turned off a RAID 0 array can provide faster reads and writes for many parts of the workflow were absolute sustained reads and sustained writes are required.

               

              Again, I am not even saying here though shalt use RAID 0. I am stating however, that it will probably serve you best and you can certainly try it out both ways on you actual system before making a final decision.

               

              Either of the drive setups that you post in your latest reply sound fine to me, and the one where you write to a local 4TB that is also shared to the network on your system would probably be my choice. If I am misunderstanding the meaning of "network shared 4TB HDD" and you mean that this drive would actually be on another PC or server on your LAN, then this would be a bottleneck and you would be better off completing the render output and copying the file to the 4TB while you do something else.

               

              Regards,

               

              Jim