-
1. Re: Scratch Disk Questions
Harm Millaard Oct 7, 2009 7:17 AM (in response to ambisinistrous)What disk setup are you contemplating? Number of disks, interface, size, speed?
-
2. Re: Scratch Disk Questions
ambisinistrous Oct 7, 2009 10:48 AM (in response to Harm Millaard)My questions are more about general principles, with the only specific that you have 1 striped RAID array and 1 non-raided scratch drive. But I can throw out more specifics for the sake of argument.
Let's say a single 500 GB Seagate Barracuda 7200.12 (7200 rpm, SATA) for scratch/pagefile/renders. Then 2x 1 terrabyte drives from the same line in a RAID 0 array using the motherboard's RAID controller.
This makes me think of another question. What's an appropriate ratio of scratch/render storage to media storage? For instance, if you had 200 GB of HD source footage, how much scratch/render would you allocate for that?
-
3. Re: Scratch Disk Questions
JSS1138 Oct 7, 2009 3:46 PM (in response to ambisinistrous)Generally speaking, here is a good minimum/starting point.
C: OS and Programs
D: Projects/Scratch
E: Media
Don't worry about the swap.
Now from there, you can do a lot of tweaks. But the above is a good basic design for an NLE disk system.
-
4. Re: Scratch Disk Questions
ambisinistrous Oct 7, 2009 4:09 PM (in response to JSS1138)Thanks for the responses guys. I'm afraid I maybe muddled my questions. Let me try again
- Is it worthwhile to have a non-RAIDed, single-drive for scratch/renders, while using RAID striped drives for media? Or should you just keep scratch/renders on the single RAID array along with the main media files? To put it another way, if you only have 1 striped RAID array, which you use for media, does it make sense to use a separate single drive for scratch/renders?
- What's a good ratio for media drive capcity to scratch drive capacity? For instance, if you have 2 TB of media storage, how big do you want your scratch/render drive to be?
- Is it worthwhile to have a non-RAIDed, single-drive for scratch/renders, while using RAID striped drives for media? Or should you just keep scratch/renders on the single RAID array along with the main media files? To put it another way, if you only have 1 striped RAID array, which you use for media, does it make sense to use a separate single drive for scratch/renders?
-
5. Re: Scratch Disk Questions
Harm Millaard Oct 8, 2009 1:09 AM (in response to ambisinistrous)1. To get the best speed from your disk setup, you need to make sure that disk activity is spread across as many disks as you have. When editing, your OS & programs need to do housekeeping, need to load plug-ins, need to load render engines, etc. This all happens on your C drive. At the same time it needs to read your media (from the E drive) and it needs to write and read preview files and keep the media cache up to date (from your D drive). At the same time your pagefile need to be accessed. I realize this is not all the activity that takes place on all your disks, but you get the picture that a lot of different reads and writes need to occur. The more you spread the load over all your disks, the better. When you have two disks and consider striping them, realize that you can only use Aid0 to get some performance gain. However, a valid question is whether that performance gain is worth the risk of striping and if it is not better to leave them as separate disks. Of course this argument does not hold with many more disks in an array where you can maintain redundancy, then raids are a definite advantage.
2. I'm not aware of a rule of thumb for this ratio, but with storage so cheap I would use at least 500 G. Keep in mind that all mechanical SATA disks show decreasing performance when the fill rate goes up. So with current prices, you can make life easier by keeping all your disks pretty large with lots of space available. That is the reason I have 16 x 1 TB disks in my system.
-
6. Re: Scratch Disk Questions
Ferdie2 Oct 8, 2009 3:06 AM (in response to JSS1138)Couldn't you also clear out your D drive when projects are finished? Further diminishing the need for a large capacity Scratch drive.
-
7. Re: Scratch Disk Questions
Harm Millaard Oct 8, 2009 4:32 AM (in response to Ferdie2)Why are you so worried about disk space? Get several 1 TB disks and edit along. At current prices for 1 TB disks around $ 80 apiece, what is stopping you?
-
8. Re: Scratch Disk Questions
Ferdie2 Oct 8, 2009 8:18 AM (in response to Harm Millaard)I'm not concerned with disk space on the Scratch drive. But wouldn't the rendering process be faster with a 10k drive as opposed to a 7.2k drive?
-
9. Re: Scratch Disk Questions
Harm Millaard Oct 8, 2009 9:16 AM (in response to Ferdie2)Keep in mind that the Velociraptor has only 16 MB cache, whereas a 1 TB disk has 32 MB cache. Also keep in mind that on a 150 GB disk 100 GB of data amounts to a fill rate of around 70%, while on a 1 TB disk that would only amount to 10% fill rate. Comparing both disks when empty, yes, the Velociraptor is faster, but when filled up to 70% is may well be slower than a 1 TB disk filled to 10%.
Look at these charts:
The spikes are caused by the cache being depleted. With 100 GB data on the disk, the sequential read speed is around 100 MB/s
Now look at the graph from a 1 TB disk:
With the same 100 GB data on this disk the sequential read speed is around 115 MB/s
Now look at the graph for a 2 disk raid0 array with 2 x 1 TB disks:
With up to 900 GB of data the sequential read speed stays at around 200 GB/s.
And the last one, a 12 x 1 TB disk raid30 array shows these figures:
Across the board this delivers a sequential read speed of over 800 MB/s (average 853 MB/s) and no deterioration with increasing fill rates.
Although these graphs only indicate sequential read speeds, the same pattern applies to write speeds.
-
10. Re: Scratch Disk Questions
Ferdie2 Oct 8, 2009 11:44 AM (in response to Harm Millaard)Excellent data. So, to summarize:
RAID30 > RAID0 > 1TB 32MB Cache > 150GB 16MB Cache
(More q's on my own thread.)
-
11. Re: Scratch Disk Questions
Harm Millaard Oct 8, 2009 1:39 PM (in response to Ferdie2)Unfortunately, Ferdie, it is not so simple.
Raid performance depends heavily on the number of disks in the array. That is the overriding factor. Next is the raid level. That depends heavily on the raid controller used and the degree to which REDUNDANCY is required. Next is the cache memory used by the raid controller.
We all know that disks sometimes fail. We hope they do not, but in practice they do from time to time. When that happens, you lose all data on that disk, period. When you stripe disks the chances of that happening are multiplied by the number of disks in the array. When you have 2 disks in the array, chances of failure are doubled, with 3 disks, the chances are tripled, and so on. That is why I call a Raid0 an AID0 since there is no redundancy.
When you use a real Raid (with redundancy) with level 3, 5 or 6, you can have one or more disks fail and you will not lose data. The disk that went hayward will be rebuild upon replacement and it will be done in the background, so you can continue working. You can even use hot-spares. In case a disk fails, it's role in the array will be automatically taken over by the hot-spare disk.
To learn more about RAID, look here: http://www.raid.com/04_00.html
A RAID30 is a striped array of 2 or more Raid3's.
-
12. Re: Scratch Disk Questions
Ferdie2 Oct 8, 2009 1:43 PM (in response to Harm Millaard)Harm Millaard wrote:
Unfortunately, Ferdie, it is not so simple.
Yes, I was generalizing based on the scenario given. I know there are numerous variations of every aspect.
When you use a real Raid (with redundancy) with level 3, 5 or 6, you can have one or more disks fail and you will not lose data. The disk that went hayward will be rebuild upon replacement and it will be done in the background, so you can continue working. You can even use hot-spares. In case a disk fails, it's role in the array will be automatically taken over by the hot-spare disk.
To learn more about RAID, look here: http://www.raid.com/04_00.html
A couple of nits. RAID1 is redundant, isn't it? I'm not familiar with levels 2 and 3, but RAID 4 has a unique benefit. Since it has a dedicated parity drive, you can add disks to the set on the fly. I used to be a Netapp admin, and this was one of the benefits that sold us on RAID4. Nowadays, they have diagonal parity and I just don't understand anymore.
Also, you can't lose more than one disc at a time, at least not while in degraded mode. If you lose a second drive while a hot spare is being created (sometimes 8 hours or more), your volume is toast.
-
13. Re: Scratch Disk Questions
Harm Millaard Oct 8, 2009 2:25 PM (in response to Ferdie2)Correct. If you use raid levels 3, 4 or 5 only one disk can fail. If you lose more disks at the same time, bye, bye. With level 6 you can have 2 disks fail at the same time without negative impact. You can recover from a 2 disk failure in raid6. BTW, Raid4 is hardly used anymore because of it's lousy write performance.
In practice, what are the chances of 2 disks failing at the same time? I have had a problem on my server with a 6 disk raid5 array where 2 disks were showing intermittent problems. I first replaced one disk, rebuilt, then replaced the second disk, rebuilt again and had no data loss at all. On my NAS I use a 7 disk raid6 so I can have 2 disks fail at the same time (this was rather necessary since 5 out of 7 Seagates failed in the first 3 months). On my editing machine I still contemplate adding a hot-spare to each of the raid3 arrays, so that I have double redundancy, but it is slightly troublesome to fit 19 disks in my case next to two BR burners and it precludes the use of 4 SSD's in a single 5.25" slot.







