18 Replies Latest reply on Dec 22, 2010 7:34 PM by gandolf2000

    Do I need faster RAID speed?

    gandolf2000

      I have a clip that contains 4 layers of 4K R3D files (2 layers of masks + 1 layer of rotoscope + original layer) that are dynamically linked to After  Effects.  The  footage was shot in Redcode 36.

      When I turn off GPU acceleration (so, software only), the bar goes red in  the timeline.  I activate GPU acceleration, the bar goes yellow in the  timeline.  Now, correct me if I'm wrong, but that yellow line means the  clip is now GPU accelerated and it may be possible to play it back in real  time (at some resolution...maybe 1/16?).

      I play the clip.  The audio plays fine, the video doesn't play at  all, no matter what the playback resolution is set at.  It just stops  cold on the first frame.

      I check Activity Monitor.  The CPU is hitting around 40% (sometimes less, sometimes more).  The RAM has a few GB free.

      I'm on a 5.1 (2010) quad Mac Pro (3.2 GHz) with 32 GB RAM, and PP 5.0.3 on OSX 10.6.5.


      QUESTIONS:

      1  a) Is it possible that the bottle neck is the hard drive read and write speed? 

      I have an external Raid drive (4 x 2TB) connected with eSata.  I tested  it with AJA System Test.  The read and write speed are 125MB (read) and 105MB (write).  Right  now, it's limited by the eSata card I have which caps everything around  125MB.  If I get something like this:  http://eshop.macsales.com/item/Newer...gy/MXPCIE6GS2/

      I can get sustained rates of 220 MB (read) and 180 (write) from my external RAID drive (according to LaCie's website). 

      b)  If I'm editing 4 layers of 4K files at 36 MB each, in theory, I need  sustained read and write speeds of 144MB (4 x 36MB) minimum, correct?   Am I way off base here? 

      2.  Another thing that's confusing me is how the playback resolution  factors into all this.  If I set it to 1/16, what sustained read and  write speeds do I need to play back the clip (all 4 layers) in real  time? 1/16 x 144 MB = 9 MB.  Do I only need 9MB of sustained read and  write speeds to play back the clip in real time?  Is that how it works?   If so, how come I can't get it to play back in real time?

      3)  Is there a software (for OSX 10.6.5) that can measure whether or not  the hard drive read and write speeds are peaking?  Apps like AJA System  Test don't measure the hard drive when in use.  I'd like to determine  if the hard drive read and write speeds are being strained/ pushed to  the max.

       

      Any help would be appreciated.  Thanks!

        • 1. Re: Do I need faster RAID speed?
          Jim_Simon Level 8

          1a.  Seems very plausible.

          1b.  Also seems correct.

          2.    The hard drive read requirements shouldn't change.  Reduced resolution simply throws out some of the lines to make the decompression less taxing on the CPU.  But the file still needs to be read from disk at the same speed.

          3.    I don't know.

          • 2. Re: Do I need faster RAID speed?
            Butch2oc Level 1

            I have a similar setup and use BM for I/O.

            They have a great piece of software for testing drive i/os.

            Tomorrow I may get on my system and get the exact figures of drive I/o that are required for what you need.

            My internal array runs at around 210mbs bothways and I still get the

            issue you described. So I guess if your thinking of uping your array to a similar speed

            your issue may not be resolved. I did think that the yellow indicator meant it could play in RT.

            I found on a Vid Tut. from Adobe TV (I couldn't find the link) I think it was "optimising your system"

            It showed a yellow indicator on a timelione that could not play. They said this was due to the drives

            not coping. Looking forward to a faster array when they are cheaper, I think around 300mbs should

            do the job.

            1 person found this helpful
            • 3. Re: Do I need faster RAID speed?
              Level 4

              I hate to admit it but I'm replying mostly to stay in this thread and see how it works out.. which means I haven't got a clue what the answer is.

               

              --------------

              I have a clip that contains 4 layers of 4K R3D files (2 layers of masks +  1 layer of rotoscope + original layer) that are dynamically linked to  After  Effects.  The  footage was shot in Redcode 36.

              --------------

              doesnt the red do something like 800mbps using the dual hd sdi out 444 and require a lot more ( thought I heard 12 drive arrays at one point ) for live recording and playback etc ??)  anyway, I have no idea for a green bar for you, but wanna find out your solutions etc

               

               

              • 4. Re: Do I need faster RAID speed?
                Bill Gehrke Adobe Community Professional & MVP

                If you are only getting those rates there is no sense in using those drives in a RAID configuration.  Incidentally you fail to say what RAID level you are using.  Here is a four drive RAID 0 test that I ran on my PC.  It should demonstrate what a real RAID controller could do for you.

                1880 4X1TB R0 Read.png

                 

                The disk controller card you linked to is just that only a disk controller card and has no RAID capability.  And yes I agree that you could be disk limited, but I have no Mac experience and or Red experience, just a lot of benchmark experience.

                 

                I do not know any Mac disk testing software but in the PC arena we can set up a RAM disk which because it is so fast you can take a few GB  of your Red file and write it to the RAM (testing your disk read speed) and then read it back (measuring the disk write speed) and get true Red transfer rates.  Incidentally scaling the playback down is either CPU or GPU intensive.  I found better playback performance with full scale on my highend PC.

                • 5. Re: Do I need faster RAID speed?
                  gandolf2000 Level 1

                  JIM:  Thanks for clearing up the whole playback resolution thing.

                  • 6. Re: Do I need faster RAID speed?
                    gandolf2000 Level 1

                    BUTCH: Thanks for the heads up on your experience.  You're right, I was hoping to up my array speed and see if it will allow me to playback in real time.  From what you tell me, it looks like I'll have to build a new faster RAID since the max I can get with a faster controller card is around 200 MB to 250 MB.  I'll search for the vid you were telling me about and take a look.

                     


                    Yes, if you could please tell me the exact figures of drive I/O required that would be great.

                     


                    Incidentally, can you tell me what BM program you're using to test the speeds? I'd like to buy the program to test out my system.

                    • 7. Re: Do I need faster RAID speed?
                      gandolf2000 Level 1

                      ROBODOG:    I'm not sure about the HD SDI.  I don't have that set up. 

                      • 8. Re: Do I need faster RAID speed?
                        gandolf2000 Level 1

                        BILL: Sorry, my bad, I have RAID 5 on a LaCie 4Big Quadra (4x2GB).  According to the spec sheet, there was very little difference in the read and write speed between RAID 0 and RAID 5 so I opted for the redundant safety of RAID 5.  The controller card I already have is capping everything at 125MB.

                         


                        You're right Bill, the controller card in my link is just a controller card.  The hardware RAID is built into the LaCie enclosure.  I was thinking that if I buy the faster controller card, I can get max speeds between 200MB to 250MB for read/write, and was hoping this would allow me to playback the 4 layer clip in real time, but BUTCH pointed out that his system is hitting around 210MB read/ write speed and he's having the same issue.  So, it looks like I may have to build a new RAID.

                         

                        That said, Bill, if you have a moment can you please tell me the specs of your RAID configuration. Your RAID read and write speeds are off the charts.

                         

                        1)  What RAID controller card are you using?  (from the screen shot, it looks like the Areca....I wonder if that's MAC compatible....time for a google search)

                         

                        2)  What enclosure?

                         

                        3)  What hard drive models?

                         


                        Can anyone recommend a solid RAID 0 set up for OSX that can sustain read and write speeds of 300MB to 350MB? I'd be really interested in hearing about people's personal experiences.  Are there ones I should avoid?  Best ones to purchase?

                         


                        Thanks everyone for the input.  I really do appreciate it.

                        • 9. Re: Do I need faster RAID speed?
                          Butch2oc Level 1

                          Gandolf

                          Here's a screen grab of what my Black Magic I/o Card came with. Not sure if you can get it as a stand a lone, just check out their site.

                          DiskCheck.jpg

                          As you can see it gives you a good idea of what you should get out of your drives before you have to render.

                          • 10. Re: Do I need faster RAID speed?
                            Bill Gehrke Adobe Community Professional & MVP

                            Gandolph.

                             

                            Yes you are correct that my RAID controller is Areca.  That happens to be their latest series the ARC-1880.  They are about the best that you can get.  They are internal (PCIe v2) cards and interface with up to the 6 Gb/second newest drives but the drives on that test were Seagate 7200.12 series drives (3 Gb/second).  They do have cards with external connectors so if you do not have room and/or power available internally you can go external.  They are not inexpensive.  I just checked and they do have Mac drivers.  I cannot recommed an external enclosure as my drives are internal.

                            • 11. Re: Do I need faster RAID speed?
                              gandolf2000 Level 1

                              BUTCH:

                               

                              Thanks for the screen grab.

                               

                              Out of curiosity, you mentioned that despite the read and write speeds of your RAID drive, you still had playback issues in some cases.  With GPU acceleration turned on, how many layers were you using when you noticed this issue? 

                               

                              What I'm driving at is this:

                              -if you were trying to playback 4 layers of 4K R3D files (like I am),

                              -and my math is correct in that a minimum of 144MB of sustained read speed is needed to playback such a clip (without rendering),

                              -since your array can sustain 200MB read speed (which is well above the 144MB) but you're still having problems playing back the clip in real time,

                              -then am I wasting my efforts/ money in getting the faster controller card since the max read speed I can get from my external RAID is between 200MB and 250MB?

                               

                              I'm just wondering if the bottle neck is really with my RAID speed. 

                               

                              Can someone from Adobe tech support shed some light on this?

                               

                              Thanks!

                              • 12. Re: Do I need faster RAID speed?
                                gandolf2000 Level 1

                                BILL:

                                 

                                Thanks for providing me the specs on your RAID.

                                 

                                The Seagate drives you're using, are those the 1TB Barracuda drives?

                                 

                                Thanks!

                                • 13. Re: Do I need faster RAID speed?
                                  Paul Joy (UK) Level 1

                                  iStat menus is quite good for being able to check on your HD performance at any point as well as keeping an eye on CPU usage, RAM etc etc.

                                   

                                  http://bjango.com/mac/istatmenus/

                                   

                                  iStat.jpg

                                  1 person found this helpful
                                  • 14. Re: Do I need faster RAID speed?
                                    FelixUnderwood Level 2

                                    I think your main issue is that After Effects, even dynamically linked, I believe, is not supported by MPE.

                                    • 15. Re: Do I need faster RAID speed?
                                      gandolf2000 Level 1

                                      Thanks Paul.  That looks like the tool I've been looking for.  Cheers!

                                      • 16. Re: Do I need faster RAID speed?
                                        gandolf2000 Level 1

                                        Thanks Felix.  I'll look into that.


                                        • 17. Re: Do I need faster RAID speed?
                                          Bill Gehrke Adobe Community Professional & MVP

                                          Gandolf

                                           

                                          Yes they are the 7200.12 versions of the Barracuda drives as opposed to the 7200.11 series which had some major problems in their early life.

                                          • 18. Re: Do I need faster RAID speed?
                                            gandolf2000 Level 1

                                            Thanks Bill!