29 Replies Latest reply on Dec 12, 2012 12:15 PM by Glitchdog

    budget video server?

    Glitchdog Level 1

      Hi, we're looking for a budget video server (under 5k) that will accommodate two Mac edit bays (2010 MacPro and 2011 iMac, but possibly a PC down the road). We're running Premiere CS6 and don't currently need TB and TB of space. For the next several years 8TB would be fine. Ideally it would be nice to have 10gb ethernet connection, but I don't know if fiber optic is still a higher ticket item. All our footage is pretty compressed HD: AVCHD, 5D, GoPro

       

      We have great IT support here at our offices, so they would be installing the cables and system.

       

      If I'm completely naive thinking I could get this for under 5k burst my bubble, but tell me why

        • 1. Re: budget video server?
          ECBowen Most Valuable Participant

          You can definitely get that done with the 5K budget if you go with 10Gbit Ethernet or Bridged Gbit Ethernet adapters. Fiber will take you outside of the budget.

           

          Eric

          ADK

          • 2. Re: budget video server?
            Glitchdog Level 1

            Thanks! Our IT guy found this:

            http://www.techspot.com/review/528-nas-10gbe-performance/

             

            So apparently there are RAID arrays (10gig e) that are designed for a more budgetary application like the QNAP and Synology gear.

             

            I'm curious if anyone has any tried and true recommendations as some of these units don't mention anything about serving up video?

            • 3. Re: budget video server?
              John T Smith Adobe Community Professional & MVP

              According to what I have read in the past, you will not be able to directly edit video on a server... only use the server for storage and copy the files to local hard drives for editing

               

              http://forums.adobe.com/thread/969395

              http://forums.adobe.com/thread/771151

              http://forums.adobe.com/thread/851602

              http://forums.adobe.com/thread/957523

              http://forums.adobe.com/thread/858977

               

              That will all change "some time" in the future

               

              Adobe Anywhere http://forums.adobe.com/message/4682127

              • 4. Re: budget video server?
                ECBowen Most Valuable Participant

                Hosting video only means the Storage array has to handle the data rate requested by the client with their projects. These 2 units can handle easily the standard HD codecs you mentioned with the Throughput shown in the review. You can also edit directly from the network storage as long as you don't mind a slight delay from the latency of packet processing. If you do then moving files over to the client edit system is better.

                 

                Eric

                ADK

                • 5. Re: budget video server?
                  Glitchdog Level 1

                  John T Smith wrote:

                   

                  According to what I have read in the past, you will not be able to directly edit video on a server... only use the server for storage and copy the files to local hard drives for editing

                  Thanks John for all the links! I'm pretty green on this side of things, but I know there are a couple of higher end facilicities here (higher than us at least) that were doing shared editing on a SAN or something like that back in CS5.5 where all the media was located in a seperate air conditioned room. Not in our price range of course.

                   

                  ECBowen: you originally mentioned 10 gbe adapters or a bridge. I had assumed we could not take our current eSATA RAID, put it in the server room and connect it to 10 gbe as a super budget setup for our main edit bay. Guessing it could only serve to one computer. But is that actually possible with some kind of adapter or would I need, at the least, to have something like the QNAP...?

                   

                  Thank you, thank you for posting!!

                  • 6. Re: budget video server?
                    ECBowen Most Valuable Participant

                    You could connect the system hosting the E-Sata array with 10Gbit but that would be pointless. The Speed of the E-sata array would be so low compared to 10Gbit, it would be a waste.


                    Eric

                    ADK

                    • 7. Re: budget video server?
                      Alex - DV411 Level 2

                      Glitchdog wrote:

                       

                      Thanks! Our IT guy found this:

                      http://www.techspot.com/review/528-nas-10gbe-performance/

                       

                      So apparently there are RAID arrays (10gig e) that are designed for a more budgetary application like the QNAP and Synology gear.

                       

                      I'm curious if anyone has any tried and true recommendations as some of these units don't mention anything about serving up video?

                      I've seen Synology and QNAP 1GbE units working for small collaborative editing environments, so yes, they've been tried and tested.  It's awesome that they have 10GbE solutions now, thanks for that link.

                      • 8. Re: budget video server?
                        Glitchdog Level 1

                        Thanks guys!

                         

                        Alex Gerulaitis wrote:

                        I've seen Synology and QNAP 1GbE units working for small collaborative editing environments, so yes, they've been tried and tested.

                        Were any of these with a Mac?

                        So with 1gbe and video I'm guessing the footage would have to be pretty compressed?

                        • 9. Re: budget video server?
                          Alex - DV411 Level 2

                          Were any of these with a Mac?

                          In one instance - yes, although they seem to have problems with bandwidth when two editors start working, and are looking into a 10GbE solution, or something that can handle 2-3 simltaneous editing sessions.  I am still finding out what the bottleneck could be - it's possible it's not in the NAS itself but in the way it's set up, network, or on a client side.

                          So with 1gbe and video I'm guessing the footage would have to be pretty compressed?

                          It can handle the uglier (not as "pretty" compressed) formats too.   "Pretty compressed" formats (including 4K R3D) are a fraction of 1GbE, so it's ultimately the number of simultaneous streams and other variables that determine if a good 1GbE NAS can handle it.

                          • 10. Re: budget video server?
                            Glitchdog Level 1

                            Thanks Alex, very helpful!

                            • 11. Re: budget video server?
                              Glitchdog Level 1

                              Here's something quite intriguing I just stumbled upon at OWC. There's no official pricing yet, but a base setup is around 5k apparently. 48 gbs?

                               

                              http://eshop.macsales.com/shop/storage/Enterprise/jupiter/mini-SAS

                              • 12. Re: budget video server?
                                ECBowen Most Valuable Participant

                                Yes Mini SAS Storage Units with expansion capability are already available. This is just another option and all of the SAS units are faster than Network storage.

                                 

                                Eric

                                ADK

                                • 13. Re: budget video server?
                                  Alex - DV411 Level 2

                                  48 gbs?

                                  It does look interesting.  First, I thought it was a dud, but then looked up what they use:

                                  - 6G MiniSAS Expander towers or racks (48Gbs theoretical throughput in full duplex, 24Gbs one way, twice that in "wide port" configurations). No big news here - like Eric said, they're widely available from a variety of vendors.

                                  - LSI 6160 switch with wide port and extended cable length support (25m with active copper, although it only has two such active ports, the rest are passive).

                                  - Metadata controller and management software (not sure which one they use - XSAN?)

                                  - MiniSAS HBAs (not RAID) in each client station

                                   

                                  It won't be cheap fully configured for sharing - certainly more than 5K.  Cable length is still the biggest hurdle.  Also, XSAN has a lot of problems I hear, so I hope they don't use that. It might have a leg up on Fibre Channel in terms of performance per dollar - I doubt it'll be a wide margin though. We'll have to wait and see...

                                  • 14. Re: budget video server?
                                    Alex - DV411 Level 2

                                    48Gbs theoretical throughput in full duplex, 24Gbs one way, twice that in "wide port" configurations

                                    They don't seem to support 8-phy-wide ports, only 4-phy, so disregard the last part. It should read:

                                     

                                    "48Gbs theoretical throughput in full duplex, 24Gbs one way"

                                    • 15. Re: budget video server?
                                      Alex - DV411 Level 2

                                      There's no official pricing yet, but a base setup is around 5k apparently.

                                      Wonder how they arrived at that, especially for 4 nodes:

                                       

                                      "A four-workstation, 48Gb/s Jupiter mini-SAS setup starts at around $5,000"

                                       

                                      The LSI switch - $3K, any decent SAS Expander box is $1.5K and up, and that's before adding drives, cabling, MDCs, client HBAs. $5K doesn't seem realistic.

                                      • 16. Re: budget video server?
                                        Glitchdog Level 1

                                        You guys are way beyond my brain cells

                                        I thought I'd stumbled upon something new, oh well. Thanks for chimming in! I've postponed a decision for now since even the 10gbe is not in my current budget. I'll revisit this all down the road a bit. However, I think I want to go with a company that has a turnkey solution and supports the Mac like I think the Small Tree site does. Looks more pricey than 5k for a budget system though, but it might be worth it for support.

                                         

                                        Thanks again for posting guys!

                                        • 17. Re: budget video server?
                                          Alex - DV411 Level 2

                                          I thought I'd stumbled upon something new, oh well.

                                          You did.  It's a Storage Area Network based on SAS expander boxes, and it's fairly new tech.  I have yet to hear of it being installed anywhere.  I just doubt it'll be as inexpensive as they say it is.

                                          • 18. Re: budget video server?
                                            Glitchdog Level 1

                                            OK, cool! We've used a lot of the OWC products over the years, so it will be interesting to see how this Jupiter system pans out, price and all.

                                            • 19. Re: budget video server?
                                              ECBowen Most Valuable Participant

                                              I would have to agree with Alex, I just dont see how that setup can be implemented in the price range listed. That has to be the price of just the storage unit by itself without the connectivity. Networking is not tied to the traditional transmission modes. The networking protocols can be introduced on other modes providing the adapters can handle the packet processing.

                                               

                                              Eric

                                              ADK

                                              • 20. Re: budget video server?
                                                I.R Level 1

                                                For only 2 or 3 editing bays and compressed formats, gigabit ethernet would work just fine. With right hardware you can have over 110 Mb/s for each workstation (you can almost saturate theoretical 125 Mb/s Gig-E limit). ProRes 4444 2K @ 23.976 is about 40 Mb/s, for example.

                                                - Build decent PC (that will be your server), with enough RAM (say 16GB)

                                                - Add 4 port Intel network adapter that support Link Aggregation

                                                - Hardware raid card (if you want Raid 5, although I know that lot of people are using software Raid 0, with adequate backup)

                                                - Buy managed switch that support Link Aggregation

                                                - Your SAN / NAS has to be on separate network.

                                                - Install Openmediavault or FreeNas or Openindiana with Napp-it on your server

                                                - Aggregate your 4 links from server

                                                - Share via AFP/NFS …. or you can export iSCSI, but I don't think you'll need that.

                                                 

                                                I believe that whole math is much less than 5K

                                                • 21. Re: budget video server?
                                                  SimonHy Level 2

                                                  I can't give you exact specs as I'm not the one who built it, but we have a two Linux servers with about 18TBs of storage in each, connected using the zfs protocol and only gigabit ethernet, connecting to 3 to 4 edit suites at a time, and it works really well. Barely any noticeable lag, and we'll often have many many HD streams coming off at any time. It probably wouldn't handle R3d very smoothly, but avchd and ProRes422 all play fine.

                                                   

                                                  We looked at how crazy expensive a lot of the off the shelf network storage was, and decided to get something custom built. Each server cost about $5k, and when we expanded from one server and two edit machines to two servers and up to five machines connected, we had to get a gigabit switch which cost close to $2k as well. Sorry I can't help with the exact specs, but I thought I would at least reply and say it is possible.

                                                  • 22. Re: budget video server?
                                                    I.R Level 1

                                                    Actually R3D is only about 42 Mb/s, format is more CPU intensive, so that works also. ZFS is file system and logical volume manager (many would say best in the world atm). If you are using ZFS you probably have BSD or Solaris based server, because they have native kernel support for ZFS.

                                                    I forgot to mention above:

                                                    - Openmediavault, based on Debian Linux. XFS is probably best option here

                                                    - FreeNas, FreeBSD (actually NanoBSD), native ZFS support

                                                    - Openindiana + Napp-it for web administration, basically OpenSolaris, home of ZFS

                                                    - There are also other options like Nas4Free, Openfiler, Nexantastore, etc.

                                                     

                                                    Any of these are very easy to install with nice Web based GUI for administration. My personal favorite is Openmediavault, but that's just because I'm Linux fanboy

                                                     

                                                    Screen.jpg

                                                    • 23. Re: budget video server?
                                                      SimonHy Level 2

                                                      Yes, Solaris 10 apparently. Thanks. Haven't got my head around Linux very well yet. I know the other thing they did was install some very large SSDs as a cache, so once a file's been read once it'll sit in the cache for quite some time. Makes it very fast, and seems to suit the nature of editing, where you often go over a cut more than once. Plus saves wear and tear on the drives.

                                                      • 24. Re: budget video server?
                                                        I.R Level 1

                                                        Exactly, those SSDs are used as the second level ARC (L2ARC), and they are practically layer between disk and main memory. That way you are able to boost read performance. You can also add SSD for ZIL (ZFS Intent Log), to boost write too.

                                                        Anyway, back to the subject, we'll confuse original poster For only 2 - 3 - 4 workstations, installation is very easy, especially if you have good IT support (as you wrote in initial post).

                                                        I'm far for expert in IT, and I didn't have any issues building couple NAS / SAN already, with just spare parts laying around.

                                                        • 25. Re: budget video server?
                                                          Glitchdog Level 1

                                                          Thanks for all the great posts and info! I would have guessed that 1gbe would not be enough. Currently our photojournalists are using 1gbe to move and sort all their high rez RAW images.

                                                           

                                                          At the moment I"m getting about 200 Megabytes per second on a 4 drive RAID 0 (housed in the main edit room), attached via eSATA from our MacPro (2010) and I'm getting quite a few layers of H.264 playing smoothly at fullrez playback. I could use even more MB I think. So I was guessing that 200 mb was the bottom number to look for in a video server.

                                                           

                                                          Can you confirm that your editors can have multiple layers of footage, like H.264, and still get good playback with Premiere set to fullrez playback?

                                                           

                                                          I understand there is a lag anytime you start playing a clip or the timeline. I"m guessing the editors just get used to it after a while.

                                                          • 26. Re: budget video server?
                                                            I.R Level 1

                                                            I don't understand where do you get 200 MB/s ?

                                                            - 200 MB/s on 4 drive RAID0 is very low. With modern drives it should be well over 400 MB/s.

                                                            - 200 MB/s over gigabit ethernet is impossible. Theoretical limit is 125 MB/s, practical limit is somewhere about  110 - 115 MB/s.

                                                            H264 is not data intensive format. For example AVCHD is about 24 mbit/s, that's only about 3 MB/s. If you get 110 MB/s over Gig-E on each client, it is clear that you'll have CPU problem (intensive decoding), way before you saturate your Gig-E link.

                                                            Don't forget that many commercial available solutions are Gig-E based.

                                                             

                                                            Also with only 3-4 clients lag should be minimal.

                                                             

                                                            So, the setup would be:

                                                             

                                                            Server with RAID that can deliver 450-800 MB/s (and that's not an issue these days) > 4G pipe via link aggregation > Managed Switch with 8+ ports, that support link aggregation > Clients Gig-E connection (separate network, just for NAS / SAN).

                                                            For OS on your server you can use any flavour of Linux, BSD, Solaris …. you just have to avoid Windows server because it won't give you AFP (Netatalk).

                                                            • 27. Re: budget video server?
                                                              ECBowen Most Valuable Participant

                                                              Remember the E-Sata bays that are 4 bay are multiplexing off 1 E-Sata channel. What he is showing for 1 E-Sata channel with 4 drives is normal for a cheaper E-Sata card. Only the more expensive E-sata cards give you 300MB/s off 1 E-Sata channel. Yes there will be latency when you hit play or scrub through the timeline due to packet processing and how many layers in the packet. It may not be much but it will be greater than direct attached storage.

                                                               

                                                              Eric

                                                              ADK

                                                              • 28. Re: budget video server?
                                                                I.R Level 1

                                                                Thanks Eric, i missed e-Sata part completely.

                                                                • 29. Re: budget video server?
                                                                  Glitchdog Level 1

                                                                  ...it is clear that you'll have CPU problem (intensive decoding), way before you saturate your Gig-E link.

                                                                  OK my brain cells are filling up You just answered a question regarding the H.264.  Thanks for all the added info!