Of course it's different strokes for different folks!
For me, I am sooooo past tapes - forever! Tapes can go bad too as can tape drives.
My latest and greatest backup method is drive based, and for me, very fast, convenient, and with full parity RAID - reliable
- 5x3TB RAID 5 sets of drives
- put them in a 5x Supermicro hot-swap cage; I cold swap them, meaning that I turn off the PC to put the back-up drives in place and turn it off again to remove them after the backup is refreshed
- use Beyond Compare software; wonderful tool to see what is new, what has been deleted, and it is super convenient to tell it to ignore all files in cache/scratch directories, etc.; it is also very fast since you need only backup what files are new or changed
- use an Areca controller with "extra" channels from my editing RAID for the backup channels; speed is awesome with transfer rates of around 1 GB every two seconds (tape drives, eat your heart out!)
- I keep my full backup off site and have a smaller set of RAID drives to have a redundant backup (normally kept on-site) to backup the most important files
If you don't mind me asking, which 3TB drives did you go with for your large backup set?
I have a lot of 3TB Hitachi 7k3000 drives; some consumer (Deskstar) and some enterprise (Ultrastar). I do like them, but they are probably not the best option available now.
If I had to buy 3TB drives today and did not have any already, I would most likely be going with Seagate Constellation drives (ES.2, ES.3 or CS models).
HI Jim and Happy new year
I've been following your advices for a while now, thank you so much for sharing all the good know-how!
I'm in the middle of rebuilding my editing machines and i'm struggling for a reliable and fast backup method to pair with.
In my case, we use 2 workstations each having their own Areca 1880-ix-16 (for video disks in raid3)
I did the setup following Harm's detailed advices and got great results through the years, but besides that i lack knowledge in Raid array and what exactly the Areca are capable of...sorry if i ask stupid questions!
Would you care sharing some details about your backup method described above?
I'm not using the external port of the 2 Areca, so was wondering if there was something to salvage here
Idealy, a nice External enclosure in Raid5 connected to the 2 Areca external SFF-8088 would give great speed...Is it even possible in this case?
Also, could one enclosure be connected through 2 SFF-8088 to the 2 workstations Areca? I know a NAS can dothat as i've been using one until now, but the speedisn't that great.
Any advice would be much appreciated
LTO is the long term archive and perfectly fine if you duplicate tapes. LTO 4 to LTO6 drives will work for this with LTO 6 having larger tapes. LTO's transfer around 100 to 150MB/s so you get about the same transfer as a single drive +.
I am considering this set up:
- My workstation on which I have the projects I am actually presently working on on either a fast SSD / HDD RAID 0 set up (and back these up to my external HDD RAID 5 box)
- A RAID 5 external HDD solution connected to my main workstation for backup of my present editing projects, as well as for both local storage and first location backup combined in one package. Because the RAID 5 already allows for one or two disc failures to recover from. To keep things simple I will buy a preconfigured RAID5 set like the Promise Pegasus or the LaCie Big series. Either Thunderbolt or USB, because I do not need huge speed since these disks are not going to be used for my actual video editing projects. They come from 8 to 20 TB boxes, so if one would fill up, I just simply buy one more and continue to work. This seems to be the simplest solution to me.
- I may actually connect one more of these boxes, but then the ones especially designed for RAID 0, to use as editing discs, then ofcourse I would go Thunderbolt for actually having the fastest data throughput.
- A USB or SATA connectable tapedrive, like something from the HP StorageWorks or HP LTO series. From there a will make the secondairy copies of my various content and project files kept on the RAID 5 boxes that I wish to keep double backed-up. And these tapes I will just store on a location different than my office, probably just at my home.
Here I will keep my back up tapes as ultimate disaster recovery backup solution. I am not so much concerned with speed here, because all my files do not need to be back within a few seconds. And I cannot imagine that anyone would need an instant backup in seconds of every file they ever produced. So I am really considering tape as an attractive secondary off-location backup solution like you as well.
And I would really like to hear some more opinions from other users here on this topic.
Essentially the key to tape archive is an intermediate storage setup to dump data to for Tape archive. The reason is data moves between drives much faster than tape when dealing with raid arrays/networks. The Intermediate is the repository to hold data that can backup on tape over time without effecting production. That is why the intermediate storage is good.
Yes, tape cannot be used for real time instant backup solutions, it needs to run slower, for example as a secondary backup running overnight copying the work of the working day that just passed, and which has already been backed up from the workstation editing disks to the first and much more faster (almost real time) backup layer of the External RAID 5 Storage & Backup solution directly connected to the workstation. That is at least how I envision this type of backup setup which I am planning for myself.
Although, how ECBowen stated: 100-150MB transfer rate; to me this still looks really fast enough for a secondary backup layer solution.
Sans Digital makes a great 8x TowerRAID device that works pretty much exactly as you describe. You can connect SFF-8088/SFF-8088 from two separate computers to the two connectors on the TowerRAID. I have personally used this setup for simultaneous connections to two 4x drive arrays from a server and a workstatoin that both had Areca's w/ the external SFF-8088 port.
Speed is awesome; much faster than NAS, eSATA, USB 3.0, etc.
PS - Sorry, I almost missed your post. You maybe should have created a new thread for your question.
Thank you for your insight !
Sans Digital looks great indeed. In this configuration, would you recommend a TowerRAID model with its own Raid controller or use the Areca from the workstation as Raid controllers through its external ports?
PS: you're right i should have create a new post, but i felt it wasn't related to Premiere per se
Definitely the Areca! Sans Digital's bundled models ship (or at least they did 1 year ago) with Highpoint controllers, which are close to the bottom of the feed chain as far as hardware controllers go. Areca's cards are 7/8's of the way to the top of the feed chain (gotta leave some room for ATTO ) and you already have two! Don't worry about PCIe bandwidth for a single card, you will not be saturating them with rotating drives.
Additionally, if you have any internal channels free you can route those "external" as well; here's a link to my Dell Super Titan post which shows some photos of the cables and two of the Sans Digital 8x TowerRAID appliances.
Thank you JIm!
Good to know about the controllers!
I think i'll use one 8X TowerRaid with two 4x drives arrays, each one connected/controlled through each SFF-8088 available .
Oh yeah, THAT post...must say it's a sweet setup you got there !!!!
Something is bothering me though...
if the external backup array is controlled by the Areca from the workstation, and the workstation fails big time, how can i access it to recover the data ?
Correct, you would need a working workstation with a working Areca controller card (should be similar vintage firmware, but does not need to be an identical Areca card) to see the data. You said that you have two workstations with Areca controllers, so assuming that one workstation is still working, you could use the unused 4 ports on that workstation to get at your backup data.
In my way of thinking, this is still an excellent option.
- Intel RAID motherboard solutions are much less robust than Areca (or competing expensive hardware PCIe solutions)
- Tape and drives are often flaky and tapes written on one drive may not be as happy restoring from another due to subtle head alignment issues, etc.
I din't think it would work with a different controller, wow those Areca are some powerfull beasts
Thanks for the insight Jim!