13 Replies Latest reply on Sep 13, 2011 8:05 AM by Neofilm

    Hard Drive Setup for Source Files, Media Cache, Project, Render, etc...

    Neofilm Level 1

      I am working to set up the following drives:

       

      C: SSD for OS and Programs

      D: RAID 0 (Three SATA II F3's) for Premiere Projects, where I render too, and Encore Projects

      E: RAID 0 (Two SATA III WD Caviar Blacks) for Source Files, media cache, and preview files (if i can get the Marvel controllers/SATA III to RAID that is)

       

      External HD USB 3.0 for backing up projects

       

      Should I be doing this different?

       

      Does anyone think it would be better to use SDD drive(s) instead of HDDs? I

        • 1. Re: Hard Drive Setup for Source Files, Media Cache, Project, Render, etc...
          Harm Millaard Level 7

          Put your source files on the D drive, together with your projects. Put the static pagefile on E. With these slight modifications, you distribute your disk accesses somewhat better, giving you even better performance. Just be sure you have regular backups of your important data.

           

          If you want to use SSD's instead of HDD's, you will not notice any performance gain, but you will effectively empty your wallet. I'd rather have you donate that money to Bill and me for further improvements of the PPBM5 Benchmark

          • 2. Re: Hard Drive Setup for Source Files, Media Cache, Project, Render, etc...
            Neofilm Level 1

            Harm, thanks!

             

            For clarification:

            Are you saying that I should leave everything like it is, except put the Source Files in the same place as the project files?

             

            If so, then it would look like this:

            C: SSD for:

            • OS and Programs

             

            D: RAID 0 (Three SATA II F3's) for:

            • Premiere Project
            • Where I render project too
            • Encore Project
            • Source Files (Per Harm's suggestion)

             

            E:  RAID 0 (Two SATA III WD Caviar Blacks) for:

            • Media cache
            • Preview files
            • Pagefile

             

            I have 24 GB of DDR3 RAM. I didn't alter my pagefile at all. When you said "static pagefile on E", what is the best way to do this? If I recall correctly, when I go to set this I can set the pagefile for all the drives. Should I set all drives to "none" except the E? And, how big should I set the pagefile to?

             

            And, this maybe my last question. Do I need two RAIDs?  That is, does the pagefile, media cache, and preview files need to be on a RAID?

             

            I don't know which RAID will be faster:

            1. RAID 0 with three SATA II 3Gbs Samsung OR
            2. RAID 0 with two SATA III 6Gbs (Actually stripped RAID setup through Windows 7)

             

            If I don't need two RAIDs set up then I'll have more backup options. However, that may not matter because I can back up automatically with my external WD. Also, I don't know how well the SATA III will RAID set up through Windows 7. Forum's didn't seem too favorable to the Marvel controller. I don't know much about all that though...

             

            I'm just trying to increase performance of course when editing, adding affects (to H.264), and increase rendering speed...

            • 3. Re: Hard Drive Setup for Source Files, Media Cache, Project, Render, etc...
              Neofilm Level 1

              Regarding page file, and how to set that up best, should I partition a peice of my E drive, maybe 50 GB, and use that?

              • 4. Media Cache, Preview, and Pagefile: Read or Write Speed more important?
                Neofilm Level 1

                Also, regarding my E: drive for media cache, preview files, and pagefile (assuming all this is supposed to be on a different drive). I'm trying to figure out if I should get an SSD for that or an HDD. It will depend on what's more important, read or write speed. If read speed is more important than I can get an SSD for under $200 that will be really fast. If it's about writing then my WD's will be better. so, which is more important for these things, writing or reading?

                • 5. Re: Media Cache, Preview, and Pagefile: Read or Write Speed more important?
                  Harm Millaard Level 7

                  A three disk raid0 is faster than a two disk raid0, even if the latter is SATA3. With the modification I suggested, projects and media on one logical drive D, you really only need to backup your D drive. Everything on your E drive is expendable and will be recreated if necessary. Setting up your pagefile as static with a fixed size of say 12 GB, considering your 24 GB of RAM, is done from the control panel. It requires a reboot to take effect and ideally you want to clean the E drive before doing that, since then your pagefile will be the first file on that logical volume, located on the fastest part of that volume.

                  • 6. Re: Media Cache, Preview, and Pagefile: Read or Write Speed more important?
                    Neofilm Level 1

                    We're almost there Harm, thanks! I'm about to indicate this as being "Answered"

                     

                     

                    More explicitly, are you suggesting it would look like this?

                     

                    C: SSD for:

                     

                    • OS and Programs

                     

                    D: RAID 0 (Three SATA II F3's) for:

                     

                    • Premiere Project
                    • Where I render project too
                    • Encore Project
                    • Source Files (Per Harm's suggestion)

                     

                     

                    E:  RAID 0 (Two SATA III WD Caviar Blacks) for:

                     

                    • Media cache
                    • Preview files
                    • Pagefile

                     

                    Is two RAIDs more ideal? That is (provided the above configuration is correct); does the pagefile, media cache, and preview files need to be on a RAID?

                     

                    Should I partition a piece of my E and use that for the pagefile or pagefile, media cache, and preview files? If yes, which? and how much of a partition should I create?

                     

                    When setting the pagefile, should I set all the other drives to "none" (except the E of course)?

                     

                    Lastly, regarding my E: drive for media cache, preview files, and pagefile (assuming all this is supposed to be on a different drive). Would I be better off using an SSD for that? Or should I keep my E: drive HDD with RAID 0? I suppose it will depend on what's more important, read or write speed. If read speed is more important than I can get an SSD for under $200 that will be really fast reads. If it's about writing then my WD's in a RAID will be better. So, which is more important for these things, writing speed or reading speed? If the SSD would be a solution for the extra files then I’ll have this set up:

                     

                     

                     

                    C: SSD for:

                     

                    • OS and Programs

                     

                    D: RAID 0 (Three SATA II F3's) for:

                     

                    • Premiere Project
                    • Where I render project too
                    • Encore Project
                    • Source Files (Per Harm's suggestion)

                     

                    E:  SSD for:

                     

                    • Media cache
                    • Preview files
                    • Pagefile

                     

                    F:  One SATA III WD Caviar Black for:

                     

                    • Extra Backups and stuff

                     

                    Thanks Harm, I'm about to have an unprecedented editing experience I think! Finally, after more than 7 years!

                    • 7. Re: Media Cache, Preview, and Pagefile: Read or Write Speed more important?
                      Harm Millaard Level 7

                      1. Don't partition your hard disks. The pagefile will go nicely on your E drive without partitioning.

                       

                      2. You might consider exporting to your E drive, since all the reads are from your D drive, so the accesses are more evenly distributed. Of course that entails that you also need to backup you exports.

                       

                      3. Despite all the posts/rumors I still think SSD's are overrated at the moment and I would prefer a two disk raid0 over a single SSD. But that is me.

                       

                      If you need a large backup disk, consdider the Hitachi 7K3000 series, possibly in 2 TB size. Affordable, fast and reliable, no longer worthy of the 'Death-star' nomen.

                      • 8. Re: Media Cache, Preview, and Pagefile: Read or Write Speed more important?
                        Frédéric Segard Level 2

                        If the SSD is a really fast one, especially on writes (like the Corsair Force GT), putting the pagefile on it could be a very good option. But if it's a standard SSD with poor write performance, put it on the HDD RAID.

                        • 9. Re: Media Cache, Preview, and Pagefile: Read or Write Speed more important?
                          Harm Millaard Level 7

                          Frederic,

                           

                          Bill has started testing with the latest (and greatest) generation of SSD's with the newest Sandforce chipsets and still encountered serious W problems and degradation after several runs of tests. Initially results were great, but after running several tests, performce degraded by more than 60%, requiring a secure erase to get back to old performance levels. Not encouraging, I think.

                          • 10. Re: Media Cache, Preview, and Pagefile: Read or Write Speed more important?
                            Frédéric Segard Level 2

                            Ouch!  That bad? Not encouraging at all... very disappointing indeed. Thanks for sharing the test results, Harm. I'll stick to keeping the SSD C: drive to OS and apps only. And I'll configure the pagefile on a secondary RAID set then. Currently planned RAID config for 12x 7K3000 1.5TB... Set1 for project and media (8xRAID3); Set2 for cache, previews, exports and pagefile (4xRAID0).

                             

                            On a side note: I should be receiving my beast in about 10 days.  All the specialized parts (like the Areca), take some time to come in! I can't wait, because I'm really tired of editing on my current piece of crap that takes about 4 times more time to get the job done!

                            • 11. Re: Confirming asset allocation/setup & workflow
                              Neofilm Level 1

                              Thanks again Harm,

                               

                              We're sooo close:

                               

                              So you're suggesting that I render out to the E: drive? (to make absolute sure we're talking about the same volumes please confirm the entire set up at end is correct).

                               

                              And, to complete the workflow I'm now curious what to do if I use Dynamic link through Encore, where do the files transcode? How do I set that to E so I may use/take advantage of "evenly distributed accessess" (as I would benefit in  your suggestion of rendering out presummably of Premiere onto E)? In other words,  what would I do/set in Encore to take advantage of my system setup (as described below ) when using dynamic link?

                               

                              Or do you even use that dynamic link workflow? If not, what is your workflow (render from premiere interface or adobe encoder? or transcode in Encore/use dynamic?)?

                               

                              My workflow has been rendering from Premiere and then importing into Encore already encoded because of issues doing it any other way. But, sometimes there's problems keeping all the chapters which is really irritating. I'm hoping these RAIDs and new setup will prevent those little issues and speed up rendering time (and improve editing & processing effects). I also hope that I can start using After Effects and not get frustrated with performance issues, but I'll go there later...

                               

                              Oh, and I just remembered, I recently changed my workflow (before I started with these RAID's) to where I've been making rendering out an "image" from Encore because of burning issues, mostly when authoring/building/burning more than one DVD.Then I started using 3rd party software to make the DVD's from the image. I hope I don't have to do all that anymore...

                               

                              Please confirm that this is correct:

                              C: SSD for:

                               

                              • OS and Programs
                              • No Pagefile

                               

                              D: RAID 0 (Three SATA II F3's) for:

                               

                              • No Pagefile
                              • Premiere Project
                              • Source Files for Premiere Project (Per Harm's suggestion)
                              • Encore Project

                               

                               

                               

                              E:  RAID 0 (Two SATA III WD Caviar Blacks) for:

                               

                              • Pagefile (setup first, 12 GB since I have 24 GB RAM)
                              • Media cache
                              • Preview files
                              • Project Render (e.g. MPEG2-DVD, H.264, etc...)/Source Files for Encore Project

                               

                              Thanks again!

                              • 12. Re: Confirming asset allocation/setup & workflow
                                Harm Millaard Level 7

                                Please confirm that this is correct:

                                C: SSD for:

                                 

                                • OS and Programs
                                • No Pagefile

                                 

                                D: RAID 0 (Three SATA II F3's) for:

                                 

                                • No Pagefile
                                • Premiere Project
                                • Source Files for Premiere Project (Per Harm's suggestion)
                                • Encore Project

                                 

                                 

                                 

                                E:  RAID 0 (Two SATA III WD Caviar Blacks) for:

                                 

                                • Pagefile (setup first, 12 GB since I have 24 GB RAM)
                                • Media cache
                                • Preview files
                                • Project Render (e.g. MPEG2-DVD, H.264, etc...)/Source Files for Encore Project

                                 

                                Correct.

                                 

                                When I have finished my PR projects, I use DVD-HQ  Bitrate & GOP calculator to determine the bitrate settings and export to MPEG2-DVD or H.264-BR depending on my delivery format, but also because I always export 5.1 sound instead of 2.0 AC3. I only make my chapter points in Encore and do not bother to try that in PR, because of the limitations of long GOP. Encore has those same limitations, but I prefer to do it in Encore, because then I can easily adjust them and make the poster frames to my liking.

                                 

                                So, in my workflow, I have my source media and project on the D drive and export the MPEG2-DVD or H.264-BR to the E drive. Then open the Encore project from my D drive and import the assets on my E drive.

                                 

                                From Encore I export the image file and then use Nero to make multiple copies, because with Nero I can use multiple burners, so creating 50 copies goes twice as fast with two burners or even 4 times as fast with four burners.

                                • 13. Re: Confirming asset allocation/setup & workflow
                                  Neofilm Level 1

                                  Great, thanks again Harm, I believe this has been answered  now:)

                                   

                                  Man, that's awesome to know about Nero. I used to use it 6 or so years ago and had to quit using it because it had some issues. But, surely it's better now. I'll have to try multiple burning with Nero sometime, I've always wanted to do that!

                                   

                                  I'm curious about making chapter marks in Encore. In my opinion it's SO much easier to do it in Premiere because I usually have chapter marks where the split in clips are. And, in a big project with a lot of chapter marks Encore seems to be tedious to work in. I really hate working in the timeline in Encore, it's horrible really. The only thing I don't like about doing it in PR is the chapter marks don't follow the placement within the timeline (UNLESS YOU KNOW OF A WAY;) when you move the contents of the timeline around forcing me to have to move the chapter marks. It's not a big deal though, because I usually only have to deal with that when I want to make a change after having thought I was done.... (AKA, a mistake;).

                                   

                                  So, it sounds like you don't use the "Dynamic link" feature... too bad, it's a great idea, but like you I don't use it yet either, seems to be a bit quirky...