-
1. Re: Generic Guideline for Disk Setup
Colin Brougham Jun 18, 2010 6:42 AM (in response to Harm Millaard)Harm,
Thanks for posting this; very interesting. It's now making me question if I did the right thing in setting up my hard drives!
I must admit I didn't do much research on this aspect (because I thought I was going to have burn-in time, but I had to hit the ground running with the new system!), but I think I have enough drives to squeeze out a decent bit of performance.
I see that you're recommending RAID 3 or 5; I had setup my 4x 1TB Samsung F3s as a RAID 10. Did I do not such a smart thing? It looks like I could probably better utilize these for speed and storage space by perhaps changing up the way they're set up. I have been running RAID 0 with my three-year-old HP workstation for my media drives (2x 320GB) and haven't had a problem, so I really don't have any qualms about RAID 0, especially with proper backups, as you suggested. I'm using the built-in ICH10R RAID controller on the mobo for the moment, as I don't have the change for a separate hardware RAID controller right now, and I believe it only provides RAID 0, 1, 5, and 10.
The OS and programs are on the VelociRaptor, and I have a 500GB/7200RPM Hitachi for projects, graphics, and the like. I usually export to the "Projects" drive, and I currently have the page file there.
Any quick thoughts about how to make it better?
Thanks again for the thought-provoking post...
-
2. Re: Generic Guideline for Disk Setup
Harm Millaard Jun 18, 2010 7:43 AM (in response to Colin Brougham)In my Raid article, I described a Raid10 as:
RAID10
The RAID level for paranoids in a hurry. It delivers the same redundancy as RAID 1, but since it is a multilevel RAID, combined with a RAID0, delivers twice the performance of a single disk at four times the cost, apart from the controller. The main advantage is that you can have two disk failures at the same time without losing data, but what are the chances of that happening?
If you have not had any qualms about using a 3 disk raid0, than I would suggest to use a raid5 with your 4 disks. Gives you almost the speed of a 3 disk rai0, but with protection against one disk of the four failing without data loss.
-
3. Re: Generic Guideline for Disk Setup
Colin Brougham Jun 18, 2010 7:52 AM (in response to Harm Millaard)HA! Good call... yeah, I guess I was being needlessly twitchy. I've always got backups of my media before it even hits my edit drives.
I was running just a two-disk RAID 0 before, which was ample for my needs at that time--of course, my system didn't know the difference, because it couldn't eat that fast. The new box has a bit more of an appetite, and everytime I walk out of the room, I swear I hear, "Feed me, Seymour!"
Between the choice of a 3-disk RAID 0 + 1 extra for whatever versus a 4-disk RAID 5, you'd say take the slight performance hit, and go for peace of mind? Makes sense to me...
-
4. Re: Generic Guideline for Disk Setup
Harm Millaard Jun 18, 2010 7:58 AM (in response to Colin Brougham)Correct and with the advantage of a simpler organization of all your files and directories. Now, if you have a large library of stock footage and music and lots of downloaded files, stuff you don't need very often, it makes sense to dedicate a separate drive for that purpose. This is why I described the topic a Generic Guideline.
-
5. Re: Generic Guideline for Disk Setup
Colin Brougham Jun 18, 2010 8:03 AM (in response to Harm Millaard)Good tips, and an excellent rationale. I'll make it so. And I'll stop polluting your thread now
Thanks, as always, Harm...
-
6. Re: Generic Guideline for Disk Setup
BrownCow Productions Jul 8, 2010 5:25 PM (in response to Harm Millaard)Hi Harm:
Thanks for the awesome guidelines and posting on all things hardware. You 'da man.
I now have a 4-disk CS5 setup (upgraded from a 2-disk CS4 setup), and will follow your guidelines. I think I understand the rationale of it all (spreading disk read/writes over multiple, low-filled physical HDDs).
A few questions:
- Does the media cache files (assuming these to be indexes, peak files, etc.) AND the media cache database go on the same drive? (E: in my case)
- Does the media (video files, stills, music, etc.) AND the *.prproj go on the same drive? (D: in my case)
It looks like Project Settings>Scratch Disks sets up the Captures, Video and Audio Previews in CS5. And Preferences>Media sets the "Media Cache Files" and "Media Cache Database" locations.
I have a WD Raptor 10k rpm drive for my OS/Program drives and a pair of 1TB WD Black 7,200rpm drives for my E: and F:. I have a motley mixture of SATA drives, all 7,200rpm of various sizes for my legacy projects & media.
I also have a software mirror drive (a Green WD 1.5TB) that backs up necessary files every morning via MirrorFolder software. So I guess I technically have a 5-disk system.
Thanks in advance! -
7. Re: Generic Guideline for Disk Setup
Harm Millaard Jul 8, 2010 11:34 PM (in response to BrownCow Productions)The table shown is about your normal disks, not about backup disks, network drives or NAS/SAN drives. In your case I would say you rank in the 4 disk category, because of the motley of various disks you have, even though this motley may be about 8 or more disks.
-
8. Re: Generic Guideline for Disk Setup
BrownCow Productions Jul 9, 2010 7:07 AM (in response to Harm Millaard)Fair enough, about the disks numbers.
Can you provide any help on the two questions I asked, please?
-
9. Re: Generic Guideline for Disk Setup
Harm Millaard Jul 9, 2010 7:30 AM (in response to BrownCow Productions)That is what I would do. So yes to both 1 and 2.
-
10. Re: Generic Guideline for Disk Setup
BrownCow Productions Jul 9, 2010 8:52 AM (in response to Harm Millaard)Thanks, Harm, I appreciate it!
-
11. Re: Generic Guideline for Disk Setup
poppabear69 Jul 9, 2010 7:54 PM (in response to Harm Millaard)Hi Harm,
Thank you for the disk guidance. Per your advice, I am going to reorganize my disks and get rid of my RAID 0 I have three 1TB disk, and one 320 GB disk.
Which disk should I use the 320 GB one? I'm thinking Pagefile, Media Cache.
I have an ASUS P6X58D motherboard and I can use two SATA 3 drives. Although it seems like current SATA 3 drives aren't much better than the SATA 2 drives, when I get SATA 3 drives, which ones would benefit the most for performance?
Aloha,
Roy
-
12. Re: Generic Guideline for Disk Setup
Harm Millaard Jul 10, 2010 12:39 AM (in response to poppabear69)The 320 for Pagefile and Media Cache is fine.
SATA III is theoretically faster than SATA2 in terms of bandwidth. However no conventional disk uses the full SATA2 bandwidth, so it is a fable that SATA III is faster in practice. The only advantage of SATA III disks is the larger 64 MB cache and it makes no difference at all whether such a disk is connected to SATA III or SATA 2.
-
13. Re: Generic Guideline for Disk Setup
poppabear69 Jul 10, 2010 1:15 AM (in response to Harm Millaard)Hi Harm,
Thanks for your advice.
Aloha,
Roy
-
14. Re: Generic Guideline for Disk Setup
stenapp Jul 12, 2010 9:41 AM (in response to Harm Millaard)Perfect timing for your posted guidelines, Harm. My machine in for a rebuild after I lost TWO RAID 0 arrays at once
(power outage while I was working, and with no UPS backup. That won't happen again. But I digress.
I'm a little confused about how you partition the RAID arrays into logical drives in your generic setup. If you dedicate different parts of the edit task to different physical drives when working with 4 drives or less, wouldn't you do the same with logical drives in a multi drive array?
At the moment I am looking at 4 1TB drives, 2 640 GBdrives, and a 500 GB drive for a new build. All drives are 7200 rpm drives from the same manufacturer. I also have a bay for swapping drives for backup.
In addition to the MoBo and a cheap Promise drive controller, I followed your suggestion and got an Areca 1210 plus the Areca battery backup. This Areca card only supports 4 drives. Now I have to assign the drives, and convince the guy actually doing the build to do what I want.
At this point I am no longer sure that what I originally had it in mind to do is still the best thing for me. I originally thought that I would use the two 640 GB drives in a RAID 1 as the system and programs drive. Then I was going to use one of the 1 TB drives as the drives for project files and source media, and three of the 1 TB drives in a RAID 5 for everything else.
But your suggestion of only having two logical drives and maximizing the number of drives in the RAID array is forcing me to rethink that. The Areca card I got is only good for supporting 4 drives. Should I do four drives on that card? in a RAID 3 or a RAID 5 (not sure which)? Or use three 1TB drives in the RAID and hold the fourth one in reserve in case of failure? Or maybe do the 4-drive array, kick out the swapping bay and add the 500 GB in for the project files?
How then do I partition and assign the different editing tasks?
I'm muddled with the choices.
Stephen
-
15. Re: Generic Guideline for Disk Setup
Harm Millaard Jul 12, 2010 9:58 AM (in response to stenapp)Stephen,
With these disks at your disposal, look at the 3 Disk configuration in the chart above.
C: should be the 500 GB disk
D: should be 4 x 1,000 Raid3
E: should be 2 x 640 Raid0 on the mobo, with the pagefile on E instead of C.
-
16. Re: Generic Guideline for Disk Setup
stenapp Jul 12, 2010 11:57 AM (in response to Harm Millaard)Thank you, Harm. That arrangement would not have occurred to me, but it makes sense when I think it through.
I will pass this on to my technician and see if I can get him to come on board with it.
-
17. Re: Generic Guideline for Disk Setup
polarsilver Jul 12, 2010 11:34 PM (in response to Harm Millaard)harm,
unfortunately i purchased my computer w/o fully understanding some of these hardware setup issues. i currently have 3 hd's--
750 gb = main os (wd re3)
2, 1 tb drives = raid 0 (western digital caviar black)
after looking at the guide in this thread and some of the remarks, i am feeling glum. what would you (or anybody) suggest here? i am not really a gearhead. it looks like the adobe guide is suggestiing i un-raid my system and have 3 non-raid drives (or add 1 more drive). i am not sure i can correctly do either option here, but i'd like to get my facts straight first. help!
alex
-
18. Re: Generic Guideline for Disk Setup
John T Smith Jul 29, 2010 9:41 AM (in response to polarsilver)A big part of your decision is based on the type of files you will edit
DV AVI type 2 at about 13Gig per hour or AVCHD at about 11Gig per hour do not need raid... see My CS5/AVCHD 1st Impressions http://forums.adobe.com/thread/652694?tstart=0
Some other file types do need raid... but only you know what you will be editing... with AVCHD I have 3 hard drives and NO raid, and all works well
-
19. Re: Generic Guideline for Disk Setup
ECBowen Jul 29, 2010 10:19 AM (in response to John T Smith)A raid 0 though does improve performance when exporting from either DV or AVCHD. That is something to keep in mind. You don't require it but it will help speed up your export times.
Eric
ADK
-
20. Re: Generic Guideline for Disk Setup
Andrew M Clark Aug 3, 2010 12:38 PM (in response to Harm Millaard)Appreciate the chart; most helpful but I have questions if one only has (4) drive bays total ....(yes, I'm referring to a Mac Pro!!).
- Should I just follow the guide using one HDD for each function as described with no raid?
- Would it be recommended to have an external setup as well?
Any suggestions would be greatly appreciated.
-
21. Re: Generic Guideline for Disk Setup
John T Smith Aug 3, 2010 12:50 PM (in response to Andrew M Clark)Someplace there is a message thread titled (something like) "to raid, or not to raid"
Using raid (mostly) depends on the type of file you will edit
For AVCHD, which is "about" 11Gig per hour of video... I do not use raid
For other files which are not as compressed, so the data being transferred from drive to memory is much more... a higher data rate is required
For someone to try and answer your question, you need to post the file type(s) and codec(s) you will edit
ADDED
Go here to read about RAID - http://forums.adobe.com/thread/525263
-
22. Re: Generic Guideline for Disk Setup
Switch7 Aug 3, 2010 1:33 PM (in response to Harm Millaard)I shoot wedding films and I have maxed out my internal raid 5 with summer projects so I picked up a new raid encloser. Here is my new configuration that I could use some advice. I just picked up an external raid enclosure Sans Digital Tr5ut-b esata and put 4 2tb WD Blacks in there Raid 5.
So now this is what I have total for Hardrives.
4x WD 2TB Black Drives (external esata raid 5)
4x WD 640GB Black Drives (Currently internal software Raid 5)
1x 60Gb Vertex SSD (OS Drive)
1x 500Gb 7200rpm Drive
This is what I was thinking, but your advice would be much appreciated.
New External Raid 5 for safe Raw Footage storage
Converting internal raid 5 to raid 0 to use as a working disk. I was thinking of transfering project footage on there while I work on it.
or
3 disk raid 0 with the 4th as my drive for exports
and the 500gb drive as my system backup drive.
I can never figure out where to direct all the folders in premiere can you tell me where exactly to direct scratch disk folders with what ever my new set up will be? Thanks. Where should the Captured Video and Audio Go? (I use DSLR footage so not really nessecary). Where should Video and Audio Previews go? "Where Should Media Cache Files" and "Media Cache Database" go? Where should my pagefile go? Is there anything I am missing if so where does it go. LOL
I know this has all been talked about, but as I read, there are so many configurations out there and I still keep getting confused for my configuration.
Thanks.
-
23. Re: Generic Guideline for Disk Setup
Andrew M Clark Aug 3, 2010 1:43 PM (in response to John T Smith)John T Smith wrote:
Someplace there is a message thread titled (something like) "to raid, or not to raid"
Using raid (mostly) depends on the type of file you will edit
For AVCHD, which is "about" 11Gig per hour of video... I do not use raid
For other files which are not as compressed, so the data being transferred from drive to memory is much more... a higher data rate is required
For someone to try and answer your question, you need to post the file type(s) and codec(s) you will edit
ADDED
Go here to read about RAID - http://forums.adobe.com/thread/525263
Yes, that would be helpful wouldn't it?!
Video will be primarily 8 bit source material; all flavors rangning from Hi-8 to AVCHD.
Audio will be 16 bit source material
Then stuff from Ae, Ai, Ps, will be utilized as well.
Also if it matters, I'm using Windows 7 64bt Ultimate.
Message was edited by: Shaluda
-
24. Re: Generic Guideline for Disk Setup
ECBowen Aug 3, 2010 1:43 PM (in response to Switch7)Question with regards to the internal raid 5. If you lose 1 to 3 days worth of work from drive/data failure, will the cost be prohibitive to you or your company?
Eric
ADK
-
25. Re: Generic Guideline for Disk Setup
Switch7 Aug 3, 2010 1:45 PM (in response to ECBowen)Not at this point no.
ECBowen wrote:
Question with regards to the internal raid 5. If you lose 1 to 3 days worth of work from drive/data failure, will the cost be prohibitive to you or your company?
Eric
ADK
-
26. Re: Generic Guideline for Disk Setup
ECBowen Aug 3, 2010 1:54 PM (in response to Switch7)I would still not recommend a 4 drive raid 0 because of the possible failure rate, but since the cost wont be prohibitive then it would not be catastrophic if a drive failed.
Eric
ADK
-
27. Re: Generic Guideline for Disk Setup
Switch7 Aug 3, 2010 2:02 PM (in response to ECBowen)I was thinking that so I could have the maximum throughput while editing. I would only make a copy of the files I am using onto the raid 0. The Project files are on the SSD Drive with a back up of those on the raid 5. So in theory if the raid 0 goes down I can still continue working from the raid 5 array. It would just be a matter of relinking to the footage on the raid 5. Correct? or am I wrong in this line of thinking.
ECBowen wrote:
I would still not recommend a 4 drive raid 0 because of the possible failure rate, but since the cost wont be prohibitive then it would not be catastrophic if a drive failed.
Eric
ADK
Thanks.
-
28. Re: Generic Guideline for Disk Setup
ECBowen Aug 3, 2010 2:06 PM (in response to Switch7)That will minimize your loss yes as long as you keep source and exported files elsewhere as well.
Eric
ADK
-
29. Re: Generic Guideline for Disk Setup
Switch7 Aug 3, 2010 2:10 PM (in response to ECBowen)Yes, but the real question is...it this the best and most efficient use of my
ECBowen wrote:
That will minimize your loss yes as long as you keep source and exported files elsewhere as well.
Eric
ADK
drives. Or would something like a raid 10 be better? I just want to find the most effective way for speed and redundancy as possible. With all these drives.
-
30. Re: Generic Guideline for Disk Setup
ECBowen Aug 3, 2010 2:18 PM (in response to Switch7)Raid 5 is the best combination of speed and redundancy. I would keep the current config and not make the change to a 4 drive raid 0 unless you needed to handle uncompressed HD files or more than 5 layers R3D material at 4K.
Eric
ADK
-
31. Re: Generic Guideline for Disk Setup
Switch7 Aug 3, 2010 2:29 PM (in response to ECBowen)Basically right now I run a core i7 920 OC to 4ghz, with 12ghz of
ddr3 1600 ram and a GTX 480. I built it for CS5 and DSLR Editing, but I am still getting stutter on playback even at 1/2 resolution when I apply a simple disolve transition. It is very frustrating I am hoping the through put of raid 0 would help that. As my system often seems to get severly bogged down once the timeline starts really getting built. I have currently been trying keep my timelines minimal and tidy to work around this. But It's not always possible. I find it is the worst if I am transitioning with alot of bright or whites in the footage. Dark footage is not as bad.
Basically the adobe TV MPE articles are a pipe dream to me so far. And my windows 7 experience index rating is maxed at 7.9 across the board all except the Disk data transfer rate which is 5.9. So that seems to by my biggest bottleneck. Now whether this mean anythign to CS5 remains to be seen but If raid 0 gets me smooth playback then I am a happy lad.
-
32. Re: Generic Guideline for Disk Setup
ECBowen Aug 3, 2010 3:13 PM (in response to Switch7)Well your software raid could be slowing the process down so it doesn't hurt to try the raid 0. That material should be able to play 1/2 or full scale on a 920 @ 4GHz without issues. Something else is going on. BTW that HDD readout for Win 7 is just for the OS drive. Unless you have a SSD or Raptor drive it's going to read in the 5.8 or 5.9 area. Unless you have files that should not be going to your OS drive going there, then you playback should not be effected by that. I would be interested in remoting into your system and seeing how you have things setup and the Adobe configuration. Something is odd.
Eric
ADK
-
33. Re: Generic Guideline for Disk Setup
Switch7 Aug 3, 2010 3:24 PM (in response to ECBowen)ECBowen wrote:
Well your software raid could be slowing the process down so it doesn't hurt to try the raid 0. That material should be able to play 1/2 or full scale on a 920 @ 4GHz without issues. Something else is going on. BTW that HDD readout for Win 7 is just for the OS drive. Unless you have a SSD or Raptor drive it's going to read in the 5.8 or 5.9 area. Unless you have files that should not be going to your OS drive going there, then you playback should not be effected by that. I would be interested in remoting into your system and seeing how you have things setup and the Adobe configuration. Something is odd.
Eric
ADK
Hmmm, I have a Vertex 60gb SSD as my OS and Project File Drive.
I have no idea how to have you remote into my system and forgive me for declining the request as you are an internet stranger and my mom always said to not open the door to strangers.
Is there anyway I can take screen shots or something for you?
Thanks for the offer to help.
-
34. Re: Generic Guideline for Disk Setup
poppabear69 Aug 3, 2010 3:44 PM (in response to Switch7)Did you have stutter when you were at full resolution?
I had a problem with stutter when I first installed my EVGA GTX 275 in MPE. It was discussed in the CS5 forum that MPE works better at full resolution. Sure enough, I haven't had a problem with stutter since I have been using full resolution. I am using the four HD configuration from Harm's recommendation and an i7-920.
-
35. Re: Generic Guideline for Disk Setup
Switch7 Aug 3, 2010 4:01 PM (in response to poppabear69)poppabear69 wrote:
Did you have stutter when you were at full resolution?
I had a problem with stutter when I first installed my EVGA GTX 275 in MPE. It was discussed in the CS5 forum that MPE works better at full resolution. Sure enough, I haven't had a problem with stutter since I have been using full resolution. I am using the four HD configuration from Harm's recommendation and an i7-920.
Yes I get the stutter at all resolutions. Unfortunately I started with the GTX 275 got frustrated so bought the Gtx 480 with minimal improvements on playback. Rendering timelines out is much faster but timeline playback stutters.
I think I might go with a 4 disk raid 10 and work from there. But the question I have regarding the guidelines is: would having all working files on a raid 10 array in one place be slower than spanning them across multiple drives like the 4 drive set up? Should I keep the footage on the raid 5, run all previews and exports and media catch on the raid 10? or have another for exports as well?
-
36. Re: Generic Guideline for Disk Setup
ECBowen Aug 3, 2010 4:03 PM (in response to Switch7)We have licensing with Teamviewer for remote access and that is how we remote in. You just need to download the client which is a stand alone file from www.teamviewer.com. We use Team viewer because it is easy for the client and safe. It requires a person on the other end to give us a user id and password to remote into your system. That was simply an offer because something was odd. I would definitely try what the previous poster said regarding full scale. The MPE and video card does handle scaling so it makes sense. That system you have should easily handle that material at full scale.
Eric ECB
-
37. Re: Generic Guideline for Disk Setup
Switch7 Aug 3, 2010 4:06 PM (in response to ECBowen)ECBowen wrote:
We have licensing with Teamviewer for remote access and that is how we remote in. You just need to download the client which is a stand alone file from www.teamviewer.com. We use Team viewer because it is easy for the client and safe. It requires a person on the other end to give us a user id and password to remote into your system. That was simply an offer because something was odd. I would definitely try what the previous poster said regarding full scale. The MPE and video card does handle scaling so it makes sense. That system you have should easily handle that material at full scale.
Eric ECB
My SSD is the OCZ Vertex 60gb. I thought the 5.9 was odd, because when I originally ran it new it was 7.4 I think. So I did some digging and looked up Trim, I supports trim and from my understanding windows 7 automatically takes care of TRIM. Is that correct?
-
38. Re: Generic Guideline for Disk Setup
ECBowen Aug 3, 2010 4:26 PM (in response to Switch7)Yes Win 7 supports Trim by default for any SSD drive that has trim support in the firmware. I was looking up that drive and reviews to see what controller is has and trim support. I could not find the controller yet but it's suppose to have trim support based on the review. Now they did not mention if they had to update the firmware for the trim support like you have to on other drives including Intel SSD. You might want to call OCZ and see if they have a firmware update for that drive. One last note was the Sequential performance results. They were in the 160MB/s to 180MB/s range which is not bad at all. I still can't find the controller though to see if it's one that has the latency issues for asynchronous commands. I will check on OCZ's website and see if they have more info. Try downloading HDTune Pro trial and see what the SSD is reading out for Read performance currently.
Eric
ADK
-
39. Re: Generic Guideline for Disk Setup
Switch7 Aug 3, 2010 4:41 PM (in response to ECBowen)ECBowen wrote:
Yes Win 7 supports Trim by default for any SSD drive that has trim support in the firmware. I was looking up that drive and reviews to see what controller is has and trim support. I could not find the controller yet but it's suppose to have trim support based on the review. Now they did not mention if they had to update the firmware for the trim support like you have to on other drives including Intel SSD. You might want to call OCZ and see if they have a firmware update for that drive. One last note was the Sequential performance results. They were in the 160MB/s to 180MB/s range which is not bad at all. I still can't find the controller though to see if it's one that has the latency issues for asynchronous commands. I will check on OCZ's website and see if they have more info. Try downloading HDTune Pro trial and see what the SSD is reading out for Read performance currently.
Eric
ADK
It's the vertex 64gb, it does have the firmeware 1.5 that supports trim. It was new enough to come shipped with that. As for the Read performance it is average of 110 mb/s Min 71 and Max 188.
But when I looked at the Health tab of HD TUNE PRO it is showing a warning in the (C7) Ultra DMA CRC Error Count. The discription says there were communication errors this may be caused by a damaged Cable.
Intereting.