Skip navigation
This discussion is locked

Generic Guideline for Disk Setup

Jun 18, 2010 5:44 AM

  Latest reply: Jeff Bellune, Aug 20, 2012 1:23 PM
Replies 1 2 3 4 ... 11 Previous Next
  • Currently Being Moderated
    Aug 4, 2010 6:34 AM   in reply to Switch7

    Ya that read speed is low. It sounds like you have an SSD with bad NAND Flash or a bad stick of ram causing the controller to send corrupted data to the drive. Also the overclock could be causing the ram errors. You should test the ram in current config and then test it at stock if you are getting errors with the overclock.

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 4, 2010 9:07 AM   in reply to ECBowen

    ECBowen wrote:

     

    Ya that read speed is low. It sounds like you have an SSD with bad NAND Flash or a bad stick of ram causing the controller to send corrupted data to the drive. Also the overclock could be causing the ram errors. You should test the ram in current config and then test it at stock if you are getting errors with the overclock.

     

    Eric

    ADK

    Well I am happy to report that after much digging and one question leading to another I found that Trim needs to be left on the login screen for a couple of hours each week for it to do it's job. If you never give your ssd a break Trim can't clean, I then found  on the OCZ site the "Wiper" clean up utility. OCZ said to use it once a month on the Vertex and other drives. I didn't think this was neccesary because "Hey I got TRIM" but thought what the heck. Lets give it a try. It took all night to run the clean but results afterwards show things back to new. The new HD Tune test is now 144mb/s up from 110mb/s and my Window experience index is up from my last test of 5.5 to 7.2. I guess that is what happens when you don't do any ssd maintanence for 4 months.

     

    Note to everyone. TRIM still needs the "Wiper"

     

    As for New Disk Set up. After testing Raid 10 and Raid 0. The Raid 10 with 4 drives giving me only 1.2TB gave me the same read numbers as the Raid 5 which was around 135mb/s that was there before it. I dont' know what the writes were as the write test didn't work. The Raid 0 gave me 208mb/s with 2.2TB of space, still don't know the write speed for that either. But when I transfer large files from the Raid 5 to the Raid 0 I am getting 115mb/s which seems to be consistent with the HD Tune test of 108mb/s I beleive this all has to do with the tranfer raid of the External Raid unit but strangely enough a tranfer test from the SSD to the Raid 0 is actually worse at around 75mb/s.

     

    How this will all effect my editing playback remains to be seen. I will have a better idea by the end of the day.

     

    So now I have all Media Footage on External Raid 5 Drive, I transfer the project I am currently working on to the Raid 0 and have Footage, Media Cache and Previews all running from the Raid 0. I have Project files on the SSD with a back up of those on the Raid 5 so there is all Project Files and Raw Footage and exported movies safe on Raid 5. I created a 20GB Partition at the begining of the Raid 0 for my Pagefile and then use my last 500Gb loner drive for all Windows related Back ups.

     

    Lets see how it all works out.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 4, 2010 9:28 AM   in reply to Switch7

    Excellent news.

     

    "Note to everyone. TRIM still needs the "Wiper"

     

    This is absolutely not correct. Wiper is a manual Trim and is meant for certain drives and is manual. Intel drives seem to have their auto trim feature functioning better so I would not use it for those. Some report issues with Wiper when the ICH is on raid or AHCI mode. You have to be careful with this and see if the drive manufacturer supports this utility. They will have it as a download if so. Some drives support the "garbage collection" instead so you don't want to use the wiper for those drive it seems. If you are not sure call the manufacturer support line and ask them what if any should you use for this.

     

    BTW those CRC errors are still at play and trim would not have an effect on those. You might want to test your ram.

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 4, 2010 10:12 AM   in reply to ECBowen

    Ok I stand Corrected. If your running a OCZ SSD, go to their site and see if they recomend you run the "Wiper" They did for my Vertex and they do for the others in that line.

     

    As for the error. I tested the ram with no OC and the error is still there. So I did more digging on the OCZ Forums and apparantely HD Tune Pro does not read SSD Drives properly. There are many people getting this error with the Vertex while using HD Tune PRo and moderators are saying it's an HD Tune Pro Error. Not the SSD.

     

    So as for now I will see how it all runs, if it all runs smoothly I will beleive them. If not I will have to find a utility that tests the ram like you say Eric.


    Thanks.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 4, 2010 10:42 AM   in reply to Switch7

    Thanks for the heads up on SSD and HDTune. I will have to check that here as well.

     

    Eric

    ADK

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 5, 2010 6:18 PM   in reply to ECBowen

    Well I am happy to report after a solid day of editing that my new configuration works amazingly well comparted to my old raid 5 setup. Everything was working as advertised. I now run all my footage and scratch disks from the 4 disk raid 0 array with project files on my ssd OS Drive and the back up footage and Project files backups on the External Raid 5 Array.

     

    The only problems I encountered was when I was trying to create a funky slideshow with JPGs I took with my 7D, I didn't scale them down so they were full 18mp resolution. After the project began to become pretty complicated it has become very unstable and keeps crashing with an Nvidia Error, saying something like my system does not meet minimum requirments. Something like that. I dug around and found others were having the same issue when trying to edit picture in their timelines. Looks like I will have to build it in AE.

     

    The other problem was stutter on 2 random clips with a disolve added. After digging more I finally found that CS5 has a know issue with AVCHD Footage that has a dissolve between 2 parts of the same clip. So I tried the work around of making a copy of the footage in explorer and importing the copy. Then using that for the second clip in the timeline. Bingo! It worked.

     

    Multiple layers are so far playing like butter. Lets hope it holds up as the sequences get more complicated as they get nearer to completion.

     

    Thanks for all your help.

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 22, 2010 8:52 AM   in reply to Switch7

    Harm, I have one question for this guide - just to be sure.

     

    I have 4 discs all SATA >= 7200 rpm. For disk 4 (disc F) you write "Previews and Export" - do you mean "Preview Renders"?

     

    Thanks.

     

    /Ulf

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 22, 2010 10:41 AM   in reply to Harm Millaard

    Thanks for clarifying it Harm.

     

    /Ulf

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 1, 2010 2:20 PM   in reply to Harm Millaard

    This post is awesome, thanks!

     

    I have a pretty awesome system set up like the 2 drive right now, but want to go to the 3 drive. Here are my questions:

     

    --EDITED--

     

    • Does the Scratch disks need to be fault tolerant? I understand an outage would interupt between saves, but I think it saves to the project file the scratch contents.
    • For editing 2 hour features of Canon footage, can I get away with a scratch drive of 64GB SSD or do I need lots bigger?

     

     

    Thanks again for the great info!

     
    |
    Mark as:
  • Currently Being Moderated
    Aug 31, 2010 6:13 PM   in reply to Harm Millaard

    Would anyone know why one of my Samsung F3s is showing up as a SCSI device in device manager?

     

    Running W7-64 with a WD Velociraptor as the OS drive + 5 x Spinpoint 1TB F3s

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 3, 2010 5:58 AM   in reply to Harm Millaard

    Hi Harm,

     

    Sorry but i don't speak english very weel. I need your help because I want to build a new PC for video and photo editing (SD and HD video).

    My problem is for hdd configuration; I don't have problem to buy 4, 5 or 6 HDD but I don't know how many TB are necessary for the scratch/temp disk and where is important to have a RAID 0.

     

    My idea is this:

     

    1 x 120 GB SSD for OS/software

    1 x 600 GB 10'000 RPM for Scratch / Temp / Pagefile

    2 x 600 GB 10'000 RPM in RAID 0 for Media / Project

    2 x 600 GB 10'000 RPM in RAID 0 for Export / Final file

     

    What do you think about this?

     

    Thank you and best regards

     

    Andrea Pedrotti

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 3, 2010 6:55 AM   in reply to Harm Millaard

    Hi Harm

     

    I've got something odd going on with 1 of my 5 F3 HDDs

     

    In device manager, 1 of the drives is showing up as a SCSI device and that drive is being referenced as a removable device through the oprion to "safely remove hardware and eject media". I acted on removal and it then disappears from the system until reboot.

     

    Any ideas?

     

    Thanks

     

    Edit: no Raid yet. All drives are plugged into Mobo sata ports.

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 4, 2010 1:35 AM   in reply to Harm Millaard

    Harm,

     

    When i'm exporting or rendering a movie to my final video, wich HDD are working? Temp and Exporting or My Project too?

     

    So you tell me tha  scratch, temp and pagefile HDD need speed and not space.

     

    I can buy this configuration, what you think?

     

    1 x 128 GB SSD for OS/software

    1 x 256 GB SSD for Scratch / Temp / Pagefile

    2 x 600 GB 10'000 RPM in RAID 0 for Media / Project / Export / Final file

     

    I don't need a lot of space because when my movie is finished I transfer them to my NAS with a RAID 5

     

    Best regards

     

    Andrea

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 6, 2010 11:09 AM   in reply to Harm Millaard

    Excellent guide Harm. I came up with a setup that I thought would work (and it does so far), and it turns out to be your 4-disk setup (technically 5 because my asset drive is RAID0), so you confirmed that I'm on the right track.

     

    Thanks!

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 7, 2010 10:15 AM   in reply to Harm Millaard

    Harm,

     

    When you talk about having 3 HD's, do they all have to reside on the computer?  Can I have a 2 drive system, with the third, fourth, etc be external?

    Thanks.

     

    David

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 7, 2010 10:20 AM   in reply to DavidKlineinTexas

    They don't all need to be inside your system as long as you have them linked in with esata. I am running a 4 disk raid 0 internally for scratch and a external 4 disk raid 5 connected with esata for footage storage. USB will be to slow. Not sure about firewire but my money would say it is also to slow. Esata will give you the same transfer rates as though it were internally connected with sata.

     

    My set has been great so far.

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 7, 2010 10:32 AM   in reply to Switch7

    Right, eSata or bust. USB and FireWire are fine for external backup drives. I use the LaCie d2 Quadras which have all three interfaces. The metal cases are built like granite and they run way cooler than a drive in a regular plastic case.

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 20, 2010 9:33 AM   in reply to PaulieDC

    I am having a terrible time getting a raid 3 setup with an areca 1680ix-12...am I supposed to include the OS disk in the raid or leave it seperate? I've only gotten as far as BIOS configuration creating a raid set. It shows all of the drives I have connected, 1 150gb velociraptor, 4 1tb samsung f3's, and 1 F3 hot spare. It's telling me that I only have 600gb total space. I really cannot figure out where to go from here, any help would be greatly appreciated!

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 20, 2010 11:15 AM   in reply to Alex_Curry

    Also I cannot get WIN 7 to install as it says there are no disks available upon startup...I have no idea what I'm doing here and getting incredibly frustrated! i'm trying to follow the manual, but it is not working properly as stated in the book

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 20, 2010 11:51 AM   in reply to Alex_Curry

    Alex, I feel your pain.

     

    Whether or not to include the OS drive in the RAID depends on what the RAID is for.  However, in the context of the rest of this thread, I am going to assume you are building a large RAID for storage of footage as you work on it.  In that case, keep the OS drive separate.  Install the OS drive on the motherboard, and the raid drives on the areca card only.  The areca card I had my tech use had a limit of 4 drives.  I ended up with three drives in a RAID 3 with a hot spare also on the card.  That shows up as 1.81 TB for the four drives, which is about right, considering one drive is the spare, one is for parity info, and the other two lose ~28 GB in the format. My OS drive is another 640 Gb.

     

    I don't know the specs on your card, but these are things to watch for.

     

    BTW my tech warned me that formatting the RAID took him nearly 20 hours, but that was necessary in order to get it to show capacity.

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 20, 2010 12:26 PM   in reply to Alex_Curry

    Alex, Windows 7 gets confused and stupid sometimes when you try to install it with all of your drives hooked up. With both installations I've done it has failed when all of my hard drives are hooked up. DISCONNECT all of your drives and only leave the C drive connected (that you are installing Win 7 on), and you should not have a problem. You can leave the power cables plugged in on the extra drives, just disconnect the SATA cables.

     

    Now, if Windows still doesn't see your C drive, hop into your BIOS and make sure you can see it there. If you can't, then Windows can't, so check your cables/connections/etc. Are you doing a RAID set for your C drive? I suggest you don't (see next paragraph), but if you are, make sure the RAID settings are correct, and make sure you have the latest BIOS from the manufacturer. You may have to use a separate PC and make a CD of the BIOS updater or however your mobo company does BIOS updates. Bottom line: you need to see your drive in the BIOS in order for Windows to see it.

     

    I didn't read all of your posts, but I'll just throw out some suggestions... you may know all this, so disregard if you do, but I'll just do a quick brain dump here: If you have a 10K drive like a Raptor, install Win 7 and the Adobe suite on THAT, it isn't worth the headache to RAID your OS drive IMO. Zero RAID isn't that much faster than a 10K drive especially if you're using onboard RAID on your mobo, and you do NOT need to RAID1 your OS drive "for protection". Instead, once you have your OS and Adobe stuff and plugins installed and set up on your C drive, you then use Windows Backup and Restore to create an image of your C drive, so you can restore it at any point in time. Your C drive, once set up, should never change. Don't use MY Documents and all that for your files, that bloats your profile and adds unnecessary stuff to your C drive. Keep it LEAN. Store your files on some other drive. I bought a WD 1TB Caviar Black for $75 on sale, gave it the drive letter J and labelled it "Junk", and that's where all my non-editing personal files and stuff goes. Great place to put your Paging file also. And make a folder called Scratch Disks and point Premiere and the other apps to use that for scratch disks. Again, keep the C drive clean. Newegg was selling the 150GB Raptors for $99 last week, and that sale will probably repeat.

     

    The RAID setup I saw mentioned should be for your assets, you need hyper-fast read speed when editing. Yes, the MPE and CUDA in video cards are amazing, but not limitless. So, RAID your assets drive, pop the scratch disk on a separate drive, keep the C drive immutable once it is setup, and you'll have a good system.

     

    With my i7, 12GB ram and 470SC card I get great performance using this drive config:

     

    300GB Raptor: C drive for Win 7 and Adobe software

    Two WD 1TB RE3s in RAID0 for assets, video and audio

    750GB Caviar Black for Project files and Encore files

    750GB Caviar Black for Photo files (I'm a photographer and keep all of my stills on a separate drive)

    1TB Caviar Black for Page file, Scratch disks and all other files that are not assets or project files, basically my junk drawer.

     

    I do break the rules and use my PC for other stuff since it's my only PC (Microsoft Visual Studio programming, SQL Server database and Microsoft Office). That should get Harm wanting to hit me, lol! SO, what I did was add another 320GB inexpensive WD drive, and when the Microsoft installer pops up with the default "C:\Program Files(86)\software name" install path, I change the C: to the drive letter of the 320GB drive so it all installs on there. When I edit, I go into task manager and stop all the Microsoft services to free up bandwidth. I'm comfortable doing that but some may not be. A separate PC for Office/etc is highly suggested, but for now what I have is what I have so I deal with it.

     

    OK, brain dump over. If you get some help from 5% of what I wrote I'd be happy! Not everyone may agree with how I set up my rig, but I hope the first part of this post gets you through installation pain.

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 20, 2010 12:27 PM   in reply to PaulieDC

    Ah! As usual Harm has the right info. Good deal.

     

    I still firmly believe you should put Win 7 on that 150GB Raptor that I NOW see you have!

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 21, 2010 6:29 AM   in reply to PaulieDC

    Thanks for all of the helpful info guys. I finally got windows 7 installed on the raptor, had to do some reformatting to be able to get my pc to see it...caused me hours of headaches! So now I am finally ready to start installing the raid. Here's how i'm thinking of setting it up:

     

    Raptor 150gb -C: win 7, program files

    4 1 tb F3's - project files, media, etc

    1 F3 for hot spare

     

    Now I am also a photographer as well...Paulie, you put all of your photos on a seperate drive correct? Would you recommend buying an external for this? I'll be running photoshop and various other programs with the adobe production suite.

     

    Time to start installing raid, I'll probably have more questions but hopefullly it goes smoothly!

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 21, 2010 8:57 AM   in reply to Alex_Curry

    Hi Alex, yeah, Ido have a separate drive for photos just because I had the room in my big honkin' Antec case, and because it helps my tired brain keep things organized. I do have a LaCie Quadra external drive hooked up via eSata as my backup drive so I always have two copies of everything, plus any photos that are really worth keeping I also backup onto a floppy.

     

    Yes, I'm kidding! (I don't have enough good photos to fill a floppy, lol)... make that a Blu-Ray disc. So, recap: Video on one RAID0 set, photos on a separate drive, LaCie Quadra eSata external for backups.

     

    SO, to answer your question, you COULD use an external as long as it's eSata (USB would be too slow for large/HDR files, would drive you batty), but you'd need another external for backups, correct? Maybe in your case it'd be better to just have a Photo section on your RAID set (I'm assuming RAID5 or 1+0 based on your 4 drives?), so you would have redundancy, therefore protected media. Is there room for a separate drive for photos? I realize you already have 5 in there now!

     

    Here's what I would do if I were you: just make a Photos folder on your RAID set, and get your rig up and running, and get "in the groove"with your workflow. THEN in the near future, you'll have a better feel for how you mentally want your rig to function and where yopu want files stored, and then the answer will come to you on how you want to deal with your photos. Then it's just a matter of copying them to the new location, whatever that is. I'm figuring that in the first three months you aren't going to be maxing out your RAID drives, so you'll have room on there, and time to think and ponder how you want your storage mapping to go. Make sense? Over the last 15 months I changed my drives around from my original plan ten times over, but now I have it just how I like it.

     

    BTW, Windows 7 has a great utility called Robocopy, and I use it for my backups all the time. It's a command-line utility, but you can make a quick batch file and keep that on your desktop, and just double-click it to run. Robocopy is blazing fast, and what it does is copy only what new files aren't in your backup location, unchanged files are untouched. And it also creates, on the backup drive, any new folders you might have on the source drive, on the fly. Let me know if you need more info on that (you may have backups planned already).

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 22, 2010 9:12 AM   in reply to PaulieDC

    Thanks for the tips Paulie...I'm going to keep my photos on the raid, why not? I can't afford to lose those either! I realize now that I need a scratch disk as well, don't know why I did not think of this sooner! I'm just using an external drive via firewire currently to allow me to start messing around with premiere and make a couple edits, but I'm looking to order another 1 TB drive solely for scratch. Are there any recommendations for this drive, does it need to be 7200 rpm or is 5400 just fine?

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 22, 2010 11:02 AM   in reply to Harm Millaard

    Alex, repeat after me: "I will never use a hard drive slower than 7200rpm. I will never use a hard drive slower than 7200rpm. I will never..."

     

     

    I'm glad Harm replied with a link to his thread. That was a great help for me too.

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 22, 2010 11:18 AM   in reply to PaulieDC

    Thank you Harm, very helpful!

     

    And Paulie..."I will never use a hard drive slower than 7200 rpm, I will never use a hard drive slower than 7200 rpm, I will never use a hard drive slower than 7200 rpm, I will never....okay i got it "

     

    Next question, why can I not find 2gb cache memory for sale anywhere? Is it just the same as ram memory only put into a cache slot on the mobo? I'm looking to upgrade from 512 mb on the areca controller...

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 22, 2010 11:50 AM   in reply to Alex_Curry

    Ooo, I didn't even know separate cache ram existed on a mobo (I have my newbie moments, believe me).

     

    Harm???

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 22, 2010 1:13 PM   in reply to Harm Millaard

    AH, good, explains that!

     
    |
    Mark as:
  • Currently Being Moderated
    Sep 22, 2010 4:50 PM   in reply to Harm Millaard

    Thanks Harm, I have the 12 port version like you recommended. Is the 667 a better choice, what is the difference?

     
    |
    Mark as:
  • Currently Being Moderated
    Oct 2, 2010 10:10 PM   in reply to Harm Millaard

    Harm, or others

     

    Do you have any experience with the Seagate 'momentus' discs, and possible in RAID, or any comment on them?

     

    http://www.seagate.com/www/en-us/products/laptops/laptop-hdd

     

    Thanks.

     

    /Ulf

     
    |
    Mark as:
  • Currently Being Moderated
    Oct 15, 2010 9:56 PM   in reply to Harm Millaard

    Planning to get an Areca ARC-1210 with four 600GB Velociraptors in RAID 0 (considering R3, but I can set up daily back ups, not too worried about the data). Would there be a RAID controller better suited for this without increasing price too much (~$300)? All reviews I read point to Areca and the next best controller I see is the ARC-1231ML at $600. I have 4 other Western Digital Black 1TB's that are not in RAID currently (don't really plan to either) and an 80GB SSD as a boot drive. Thanks!

     
    |
    Mark as:
1 2 3 4 ... 11 Previous Next
Actions

More Like This

  • Retrieving data ...

Bookmarked By (14)