7 Replies Latest reply on Sep 25, 2011 8:06 AM by PoisonNuke

    Hardware recommendations for render farm?

    PoisonNuke

      Hello,

       

      what would you recommend for a small render farm for mixed AE projects (mostly HD content, short projects with only a few seconds and layers and bigger ones with multiple HD layers and up to 15min or longer.)

       

      is it better to have more cores (AMD opteron servers with 48 cores for example) or better more performance per core but less cores (Intel Xeon servers)?

      And how much memory would be optimal?

      What can you recommend for network connection to the shared storaged, how fast shall this be?

        • 1. Re: Hardware recommendations for render farm?
          Bill Gehrke Adobe Community Professional & MVP

          AMD currently is not any good for Premiere because with this generation they do not have the SSE instruction set that Intel does.  As far as I know there are no feature built into CS5 or CS5.5 that allow it to be used with a render farm or shared storage..  For the fastest and best Premiere Pro systems look at the PPBM5 benchmarks.  (This is a Premiere Pro forum, I do not know AE and you might be better off in an AE forum)

          • 2. Re: Hardware recommendations for render farm?
            PoisonNuke Level 1

            I have originally posted in AE Forums, do not know why this question about AE was moved to Premiere.

             

            For AE it is possible to set up a render farm, where one PC controls some more other PCs and distribute render jobs.

            I hope a mod could move my thread back to AE please?

            • 3. Re: Hardware recommendations for render farm?
              Frédéric Segard Level 2

              Bill, I believe he was talking about After Effects and not PremierePro.

               

              Here are a few pointers... They are generic guidelines for server farms.

               

              As far as render farms go, It's all a question of budget. While the faster and more cores a CPU has if indeed good, you have to take into consideration how many computers you want in your render farm. You need to plan how your watch folder is setup. This common network storage place, accessible by all computers, where your AE project and media are stored, must be fast. Not so much the disk or RAID performance (although important), but NIC performance (Network Interface Card).You may ideally setup a NAS or a filer server (not part of the farming cluster), that has multiple NICs and aggregate the connections. But at a minimum, get gigabit connexions.

               

              A render farm is not all about the optimal raw speed of a computer. It's more about the quantity of computers working in parallel. So think of your budget, the cost of new computers, and think of which existing computers in your organization you can "repurpose", and steel some CPU cycles to process your renders.

              • 4. Re: Hardware recommendations for render farm?
                PoisonNuke Level 1

                thank you.

                 

                Is there any experience, how much CPU performance could be used if for example every rendering computer can fully utilize a 1GBit connection? So would a server with 2 Quadcores and guess 32GB memory enough or would even a single quadcore fit enough and it would be better to take some more single CPU servers?

                 

                so the primary question is: is it better to have some more small servers, or less servers but much bigger ones? All servers are planned to bought extra for this,

                 

                 

                so maybe would it be better:

                5 servers with Dual Quad/Sixcore Xeon and 32GB mem and 10gbit connection to storage

                or 10-15 single quadcores with 16gb and only 1gbit connection to storage?

                • 5. Re: Hardware recommendations for render farm?
                  Harm Millaard Level 7

                  Let's say you start a farm and have a budget of X. You have the choice, either one prize winning cow, or 20 ordinary cows for the same budget. Which solution will give you more milk at the end of each day?

                   

                  In your example your choice is either an expensive server, or 20 low cost PC's on a network. A large cluster of PC's over 1 Gb will be faster than a few servers over 10 Gb, at least that was my experience with the render farm of Vue, number of cows is more important that the speed of one or a few dedicated servers. I don't know the exact working of AE with the render farm, but with Vue there is a clear case of 'load balancing' going on, distributing the render load in smaller or larger chunks, depending on the speed and workload of each cow and stitching all these pieces together again for the final result. All the cows were used, but those actively used got lesser loads than those that were not very actively used.

                  • 6. Re: Hardware recommendations for render farm?
                    Frédéric Segard Level 2

                    I like Harm's analogy better then mine. One prize cow vs. 20 regular cows. The imagery is very clear. In a production environment, where volume counts, more is better then a single top notch one. A single cow can only produce so much mik per day, no matter how extraordinary it is.  But 20 can theoretically produce 20 times more (practically, maybe  17 to 18). Quantity, not quality. I'm not saying that you should get crappy hardware. Just good hardware, not high performance ones.

                     

                    As you know, it's not because you buy a computer that costs twice as much, that you'll get twice the processing power. First of all, forget dual CPU computers for a render farm, it's not worth it. Get a regular quad core. As you compare prices vs. performance (do a chart), get the CPU that gives you the best bang for the buch (just before the price curve goes up).

                     

                    I personally think 32GB is overkill in a render farm environment. 16GB is way more then enough. Remember that you're not doing a RAM preview farm (if such a thing exists), but rather, fractioning the render process to various computers. While 16GB is good, 2GB isn't. But 8GB can be an nice compromise. Remember, you're trying to drive the cost down of a computer, so you can buy many computers. But RAM is cheap, so, best bang for the buck, 16GB can be possible. Then again, it depends on other overall factors to keep the price for the computer down.

                     

                    Network speed. Yes you could get 10GB NICs and a 10GB switch. But that can drive the price up substantially. If you can get a good price on them, great. If not, stick to 1GB. Get a motherboard that has 2x 1GB NICs on board, so you can aggregate the network connections together to get 2GB. You could eventually add a quad 1GB PCIe card down the road. The computer that has the project assets should have an aggregated quad (or more) NIC inside. You want to be able to push the media as fast as possible to the render farm.

                     

                    Other things to consider... or rather, not consider. Graphic card is a non issue on a server farm. AE is all about CPU. Some mobos come with on-board graphics. Usually, it's cheaper, and consumes less power. You also want to drive the energy costs down when you multiply by X number of computers. Just get a KVM solution that will be able handle X computers. You don't need a RAID on render farm computers. A regular hard drive will do. I'd even go with a green drive that consumes less power.

                     

                    And depending on how many computers you have, to reduce the footprint, you might want to get low profile rackmount casings and put them in a rack.  Plus think of putting it in it's own room, because it'll be very noisy!

                    • 7. Re: Hardware recommendations for render farm?
                      PoisonNuke Level 1

                      thanks, very good and enlightening answer.