Skip navigation
LWRCJohn
LWRCJohn
Currently Being Moderated

How does Editing/Rendering Use Resources

Jul 10, 2012 9:18 AM

Tags: #help #adobe #sequence #premium #of #render #ae #cs6 #timeline #events #uderstand #precess

Im trying to understand the exact  sequence of events that occurs as a video is rendered in order to more precisely tune my computer for video editing.  Ihave CS6 Production Premium ( or what ever its called this release.

 

I have 3 hard drive arrays on 2 separate Adaptec RAID and a Nvidia gtx680 in a dual Xeon Hex core 2.5ghz machine w/48gb of 1600.

 

I want to set up the most efficient I/O for Adobe but still struggle to understand the way the software processes video between the various files it creates.

 

You have the RAW video files ( 1volume) you have the Rendered Files ( 1 Volume for storing the rendered video) then there is things like the media cache, and the data base.  What kind of resources do these need, how do they relate to I/O, and How much bandwidth do they use... Also getting in to After Effects, whats the difference  between the media cache and disk cache? 

 

Any help on this would be great,  again Im looking for how all this ties together as a process so I can better tune my computers RAID volumes.

 

thank you in advance.

2073 Views   20 Replies   Latest reply: Harm Millaard, Jul 15, 2012 2:49 PM
 
Replies
  • John T Smith
    Currently Being Moderated
    1. John T Smith,
    Jul 10, 2012 9:40 AM   in reply to LWRCJohn
    Report

    I'm not sure if the CS6 benchmark is fully operational, so also go to the CS5 link

    CS6 Benchmark http://ppbm6.com/

    CS5 Benchmark http://ppbm5.com/

     
    |
    Mark as:
  • LWRCJohn
    Currently Being Moderated
    2. LWRCJohn,
    Jul 10, 2012 9:51 AM   in reply to John T Smith
    Report

    Thanks for the links.  I didnt realize it had graphs.  But Id still llike a explanation of  how the process is occuring  so I can make sure that I have  the best posible setup for the reads and writes that are happening as the video is being rendered,

    thank you though for the feed back.

     
    |
    Mark as:
  • Jim Simon
    Currently Being Moderated
    3. Jim Simon,
    Jul 10, 2012 11:57 AM   in reply to LWRCJohn
    Report

    The ideal is to minimize reading and writing to the same drive at the same time.  So here's a good real world setup.

     

    C: System

    D: Media

    E: Projects

    F: Scratch/Cache

    G: Exports

    H: Images (from Encore as Masters)

     

    As you lose drives, you move the missing components up, so without H, you put Images on G.  Without G, you put Images and Exports on F.  Without F, you put Images, Exports and Scratch on E.

     

    You don't want to go any less than E:

     
    |
    Mark as:
  • LWRCJohn
    Currently Being Moderated
    4. LWRCJohn,
    Jul 11, 2012 9:15 AM   in reply to Jim Simon
    Report

    OK so  if I understand you then, the Images, Exports(what ever those would be...Same as rendered sequences?) and Scratch/Cache use the least amount of I/O and it is best to have them together than on other disks that may have Higher I/O.  Currently I have a  8 drive volume as my Source media drive where i store all my raw footage, and a 4 drive volume where I send all the rendered media to when I render the edited sequences,  I also currently keep the cache and scratch on the C drive or in their default location( where the project files is) which is usually the same drive as the Raw footage.

    So let me ask you this...

    you think if  were to break up that 8 drive volume and make it 2 4 drive volumes and put the project files and scratch/cache on one and the rendered video writing tot he other it would make my system render more quickly?

    thanks again for the advice.

     
    |
    Mark as:
  • Harm Millaard
    Currently Being Moderated
    5. Harm Millaard,
    Jul 11, 2012 9:31 AM   in reply to LWRCJohn
    Report

    See: Adobe Forums: Generic Guideline for Disk Setup

    and Adobe Forums: To RAID or not to RAID, that is the...

    and Adobe Forums: Raid Performance and Rebuild Issues

     

    Jim has not yet seen the light on raid arrays, so he has a natural tendency to advise to only use single disks.

     
    |
    Mark as:
  • Jim Simon
    Currently Being Moderated
    6. Jim Simon,
    Jul 11, 2012 1:16 PM   in reply to LWRCJohn
    Report

    Exports(what ever those would be...Same as rendered sequences?)

     

    No.  Exports are what come out of Premiere Pro.  Rendered previews are part of the Scratch settings, and are for PP's internal use.

     
    |
    Mark as:
  • Jim Simon
    Currently Being Moderated
    7. Jim Simon,
    Jul 11, 2012 1:18 PM   in reply to Harm Millaard
    Report
    Jim has not yet seen the light on raid arrays, so he has a natural tendency to advise to only use single disks.

     

    Any one of the drives I mentioned can be a RAID, Harm.  I've no quarrel with that.  In fact with tapeless media, I do often recommend the Media drive be RAID 3.  None of this contradicts the spread as indicated.

     
    |
    Mark as:
  • Jim Simon
    Currently Being Moderated
    8. Jim Simon,
    Jul 11, 2012 1:25 PM   in reply to LWRCJohn
    Report

    break up that 8 drive volume

     

    With a total of 13 drives, here's what I might recommend.

     

    C: System

    D: Media (x4, RAID 3 or 5)

    E: Projects (x2, RAID 1)

    F: Scratch/Cache (x2, RAID 0)

    G: Exports (x2, RAID 0)

    H: Images (from Encore as Masters) (x2, RAID 1)

     

    THAT config would be screaming fast!  And would offer a good measure of security for the important stuff, the media and project files.

     
    |
    Mark as:
  • LWRCJohn
    Currently Being Moderated
    9. LWRCJohn,
    Jul 11, 2012 6:57 PM   in reply to Jim Simon
    Report

    Welllllll actually I have 16 drives... all raid 5 1) OS 2)Renders 3)raw footage/projects 4)storage...

    all have 4 drives ea.

    would you care to cretail your last drive asignment to fit my box please.  Im opent to rebuilding raid arrays.  I should also state that the OS is on a 4 drive SSD array

    thanks again Jim, I appreciate your help!

     
    |
    Mark as:
  • Jim Simon
    Currently Being Moderated
    10. Jim Simon,
    Jul 12, 2012 9:27 AM   in reply to LWRCJohn
    Report

    Post the drive sizes.

     
    |
    Mark as:
  • LWRCJohn
    Currently Being Moderated
    11. LWRCJohn,
    Jul 12, 2012 5:22 PM   in reply to Jim Simon
    Report

    C: OS - 250 gb ( raid card 0)

    D: (renders) 950 gb _ thisis the 8 drive raid 5 I could break up in to 2 420 gb volumes. (raid card 1)

    E: Captures and Raw Footage - 830gb (Raid card 0)

    J; External RAID 5 ESATA Enclosure - 3.2 TB(7200rpm)

    thanks, again I wouldnt mind breaking up the D drive in to two different volumes.

    I just want the most effifient stream like high speed low drag data shover i can make!

    thanks :-)

     
    |
    Mark as:
  • Jim Simon
    Currently Being Moderated
    12. Jim Simon,
    Jul 12, 2012 6:33 PM   in reply to LWRCJohn
    Report

    No I meant the size of each individual drive (without making me do the math ).

     
    |
    Mark as:
  • LWRCJohn
    Currently Being Moderated
    13. LWRCJohn,
    Jul 13, 2012 6:13 AM   in reply to Jim Simon
    Report

    4 80 gb SSD Drives

    4Velciraptor 150gb

    4velociraptor 300 gb

    4 fujitsu 143gb SAS 15k

    4 fujitsu 1tb 7200rpm

     
    |
    Mark as:
  • Jim Simon
    Currently Being Moderated
    14. Jim Simon,
    Jul 13, 2012 9:36 AM   in reply to LWRCJohn
    Report

    OK, here's what I'd do.

     

    C: System (80GB SSD x 2, RAID 1)

    D: Media (1TB x 4, RAID 3 or 5)

    E: Projects (300GB x 2, RAID 1)

    F: Scratch/Cache (150GB x 4, RAID 0)

    G: Global Cache (for AE) (80GB SSD x 2, RAID 0)

    H: Exports (143GB x 4, RAID 0)

    H: Images (300GB x 2, RAID 1)

     

    This gives you data protection on the critical drives - System, Projects, Media and Images (Masters) - and speed on all of them.

     

    On a side note, what the hell case do you have that can hold 20 hard drives!?

     
    |
    Mark as:
  • LWRCJohn
    Currently Being Moderated
    15. LWRCJohn,
    Jul 15, 2012 10:58 AM   in reply to Jim Simon
    Report

    HAHAHAHAA  Laughing at your last comment.  Thanks fo rhtat answer...

    The Hard drives are not 3.5 FF, they are 2.5.  I use a product from thermaltake.  Look upThermalTake MAX 1542 and 1546.  Or have alook at the short video i made on it.(warning, bad camera shake )  I also have a video on the build when I made this computer.  Also the other 4 hard drives are in an eclosure that has its own raid controller and is attached via E-Sata/

     

    Enclosre: http://www.youtube.com/watch?v=ZI5gxsV-Whw

    Machine Build-Dual Hexcore Xeon 2011 with 48GB RAM: http://www.youtube.com/watch?v=X_2a7oEgDHQ

     

    thanks again for all your help sir!

     
    |
    Mark as:
  • Harm Millaard
    Currently Being Moderated
    16. Harm Millaard,
    Jul 15, 2012 12:03 PM   in reply to Jim Simon
    Report

    On a side note, what the hell case do you have that can hold 20 hard drives!?

     

    Just for your info, a simple Lian Li PC-A77 easily holds 17 3.5" disks and with different drive bays can hold 19 hard disks plus 2 BR burners.

     

    The case I'm using for my new build can hold 31 3.5" HDD's, 4 SSD's and 2 BR burners and a multicard reader. See: Adobe Forums: Planning / building a new system. Part 1

     

    Pictures are included to see what it looks like. This is without modding, but if you want you can easily add another 8 disks for 39 3.5" plus 4 SSD's plus 2 BR burners plus a multicard reader. Then you can add a pedestal for another 16 or more disks. It will not achieve the space of an Aberdeen Petarack, but for most this will suffice.

     
    |
    Mark as:
  • Jim Simon
    Currently Being Moderated
    17. Jim Simon,
    Jul 15, 2012 2:26 PM   in reply to Harm Millaard
    Report

    a simple Lian Li PC-A77 easily holds 17 3.5" disks

     

    That case is listed at one retailer as holding only "9 x Internal 3.5", not 17.

     
    |
    Mark as:
  • Harm Millaard
    Currently Being Moderated
    18. Harm Millaard,
    Jul 15, 2012 2:36 PM   in reply to Jim Simon
    Report

    You are forgetting 12 x 5.25" external bays. I currently have only 17 disks in that case. Anyway, the next one will be a bit better:

     

    Magnum17.png

     
    |
    Mark as:
  • Jim Simon
    Currently Being Moderated
    19. Jim Simon,
    Jul 15, 2012 2:44 PM   in reply to Harm Millaard
    Report

    You are forgetting 12 x 5.25" external bays.

     

    Not forgetting.  I just don't use them for internal 3.5" disks.  I use them for external 5" drives.

     

    If I have 20 internal disks, I want 20 internal 3.5" bays, where they belong.  THAT case is something I've never seen.

     
    |
    Mark as:
  • Harm Millaard
    Currently Being Moderated
    20. Harm Millaard,
    Jul 15, 2012 2:49 PM   in reply to Jim Simon
    Report

    I use them for external 5" drives.

     

    You mean you still use 5.25" hard disks or are there 5.00" drives? I never heard of them.

     

    Look at the pictures in the link I gave you, 15 3.5" bays on the CPU side, 16 3.5" bays on the PSU side without any mods and without pedestal.

     
    |
    Mark as:
Actions

More Like This

  • Retrieving data ...

Bookmarked By (0)

Answers + Points = Status

  • 10 points awarded for Correct Answers
  • 5 points awarded for Helpful Answers
  • 10,000+ points
  • 1,001-10,000 points
  • 501-1,000 points
  • 5-500 points