Adobe Community: Storage rules for an editing rig. Some basics.

Currently Being Moderated

30. Harm Millaard,

Jun 9, 2009 4:57 AM in reply to Filmemacher

Report

Offline storrage, NAS RAID

I would go for the 8 disk enclosure. You may not need to populate them all right away, but at least the backplane is 8 disk ready. Another thing to consider why a relatively large storage capacity may be beneficial is that you may need to keep one project available, while awaiting final approval from your client, but need to start your next project. And SATA storage is cheap.

Main storage, VIDEO RAID

From what you told me, the data rate of Red One 4K material is about ten times HDV. Luckily you are talking about SCSI disks and I assume about 15K RPM disks. Yes, they are expensive, but also very fast. With your relatively short timelines my guess is (this is a guess, since I have never worked with Red 4K material) that you should be OK with 6 disks in raid10, but pay attention to the cache memory on the raid controller. A 256 MB cache or a 2 or even 4 GB cache makes quite a difference. Another advantage you have is that SCSI disks do not show performance degradation with increasing fill rates as much as SATA disks.

RAID for software installation

First of all, in my case just the Master Collection uses over 16 GB. But add the OS and some additional tools and you may well end up with about double that amount. I use Vista 64 Business and of course Vista has a much larger footprint than XP, but my Windows directory is 20 GB.

RAID for page-file/scratch/renders

Using your existing 120 GB disks makes a lot of sense. Striped you will have 480 GB and that is more than enough space. Will the speed be enough? I guess so, even with older disks that may achieve a transfer rate of 60 MB/s per disk, you will increase that with a factor of somewhere around 3.2 to 3.6 by striping 4 disks. It depends on the raid controller whether you can achieve only 3.2 or even 3.6 as a speed increase over single disks.

All in all this setup makes a lot of sense for your purposes. Let me know what raid controller you are contemplating.

Be aware of the limitation of Win XP of not supporting LBA64, so you have to use MBR structures, limiting you to max 2 TB per volume. That does not apply to Vista or Server.

Hope this helps in your decision making.

|

Mark as:

Currently Being Moderated

31. Filmemacher,

Jun 9, 2009 3:38 PM in reply to Harm Millaard

Report

Thanks a lot for your advise!

Offline storage, NAS RAID

I think here we are in sync. I looked up your THECUS recommendation - and indeed you can get already here in Germany the N8800 for about 3600€ incl. VAT configured with 16TB (8 discs with 2TB each). They are other brands out there, such as xtivate, QNAP..., probably even faster but also more expensive. I wouldn't use the iSCSI functionality because of Windows XP 2TB limitation - as you stated. I would use it as simple assigned network drive so that I can see more than 2TB and I can live with the fact that a movement of data to the 900GB SCSI Video RAID probably takes 2 hours. Indeed, in such an environment a RAID level 6 (probably 5 is better) with hot spare(s) is more economical. Conservative people would probably still use RAID level 10 - however, the key arguments are that

(i) SATA storage is relatively cheap and suitable in this "buffer environment" and

(ii) Should be probably oversized in order to store a couple of recent customer projects - in case changes occur.

I think point (ii) becomes very relevant since I don't have a solution for the backup in mind. If a project takes 1-2 TB I probably need to deliver to my clients the discs, telling them to take care of the data security and after "time X" I will have erased their project from my RAID - not sure what they say.

Other 3 RAIDS

I think with the sizing we are in sync - ok, VISTA versus XP is probably a different blog. In Germany even the big production houses still use XP because of its stability. And it gets even more tricky if it comes to Microsoft's new proposed operating system. I just remember the time a couple of years ago where I got lost with my TARGA cards, where we changed from NT to Windows 2000 and from Premiere 5.1 to 6.0 but I was left without drivers for my cards.

Connection of SCSI drives

Here you made a good point - my old IBM server has an onboard 1 Channel RAID controller and I have (2) two large/full-size and (1-2) additional half-size PCI-X slots. Now, the PCI-X 133 bus is 64-bit and operated at 100 MHz and makes 800 MB/s. Ok, the bus is quite old and 20% slower than a PCIe 4x bus. One PCI-X slot is going to be occupied by the AJA card and I will probably add an additional dual PCI-X graphic card. So I have one to two PCI-X slots left and consider using a dual channel SCSI card.

Here are my thoughts: I would choose an

Adaptec 2230SLP. IT is a dual channel and it has a low-profile and physically easily fits into my server. IBM also uses these Adaptec boards - so I don't expect problems with compatibility. For the configuration I see (2) two options:

OPTION 1:

Main storage, VIDEO RAID on channel (1) one on the Adaptec 2230SLP;
RAID for page-file/scratch/renders on channel (2) two on the Adaptec 2230SLP;
RAID for software installation on onboard 1 Channel RAID controller;

OPTION 2:

Main storage, VIDEO RAID on channel (1) one on the Adaptec 2230SLP;
RAID for page-file/scratch/renders together with RAID for software installation on channel (2) two on the Adaptec 2230SLP;
I will disable onboard 1 Channel RAID controller.

Which option do you think is better? I could also go for the SAS discs or a mixed environment SAS and SCSI environment. However, I like to reuse my enclosure and some of the SCSI discs I have.

|

Mark as:

Currently Being Moderated

32. Harm Millaard,

Jun 9, 2009 3:54 PM in reply to Filmemacher

Report

Von der Weser bis zur Elbe,
Von dem Harz bis an das Meer
Stehen Niedersachsens Söhne,
Eine feste Burg und Wehr
Fest wie unsre Eichen
Halten allezeit wir stand,
Wenn Stürme brausen
Übers deutsche Vaterland.
Refrain:
|: Wir sind die Niedersachsen,
Sturmfest und erdverwachsen,
Heil, Herzog Wittekinds Stamm. :

Ich hab mal einen ersinnigen Abend mitgemacht in Kelheim mit Leuten von Hannover die dieses Lied gesungen haben....und ich hab mitgemacht mit einem Dunkles dazu. Es war damals.....

Bin aber nicht ganz sicher ob Hamburg auch als Niedersachsen angedeutet wird. Im Fall ich mich irre, bitte meine Entschuldigungen.

Back to topic:

Your setup looks very good, there is no weakness I can discern. Yes, the cost is high, but OTOH with your budget you can (relatively) easily afford that and still get a dual W5580 with 24 GB RAM and make a fabulous high perfomance machine, that makes my PC absolutely outdated, slow, and ready for replacement.

|

Mark as:

Currently Being Moderated

33. Harm Millaard,

Jun 10, 2009 3:07 AM in reply to Filmemacher

Report

I would go for option 1. The 2230 seems like an excellent choice. The only drawback is the fixed 128 MB cache, but I have not found any alternative.

|

Mark as:

Currently Being Moderated

34. Filmemacher,

Jun 11, 2009 7:29 AM in reply to Harm Millaard

Report

Thanks again for your feedback -

Well, I think the good aspect on a forum is (despite the marketing aspect) to discuss ideas, concepts before puting them into reality, which unfortunately cost a lot of money. The value of a video editing system unfortunately decays away quite rapidly and therefore, I think probably three times before I put the coins on the table and I want to see the ROI. 20 Thousand Euro for a video editing system aimed to make some money sounds like a great budget - but looking into the details it isn't (- small duplication facility for 100 DVD's or so, class 2 monitor with propper card, datasafety and storage management...).

... I had a couple of conversations with hardware vendors during the last days related to storrage management and I mentioned your idea iSCSI as well. We also discussed solutions with SATA discs only, probably used by 4x 1GB lans. As you mentioned to pair 4x 1 GB lan cables. However, the initial data transfere would be very small, probably 40 MB/sec before it catches up to about 200 MByte/sec. This latency effect is also present with iSCSI. (... your iSCSI is probably one reason to use VISTA, even though I can get tools for XP64 as well.)

I think I will go for the "3 RAID solution" as initially porposed and an additional NAS with dual link as network storage to store the ,main project and probably the once, which are a bit older in case changes ocure. In two weeks or so I will let you know how it looks like.

P.S.:

...zur Mentalität in Hamburg und Niedersachsen. Obwohl die beiden Städte nur 90 Minuten entfernt sind, für Amerikaner etwa einmal Chicago's Lakeshore Drive von South Side nach Evanston, betrachten die Hamburger die Menschen südlich der Elbe als Bayern. Die Bayern "Lederhosen" sind für die "Fischköpfe" zwar liebenswert, aber doch sehr anders - was sich auch im Bier bemerkbar macht. Übrigens sind bei uns im Norden die Fischbrötchen und der Matjes sehr beliebt - in Bayern wohl eher die Würstchen - wenn ich mal so in die Cliche-Kiste

greifen darf. Teile von Norddeutschland, das heutige Schleswig Holstein, gehörten ja auch mal zu Dänemark (...sorry) und das Alte Land "Dat Ole Land" westlich von Hamburg wurde ursprünglich von den Hollander bewirtschaftet, d.h. die habe hier z.B. die vielen kleine Entwässerungsgräben angelegt (...also ein dickes Dankeschön an die Käsköpfe). Der Mentalitätsunterschied ist wohl das Ergebnis der historischen Wurzeln zu Holland und Dänemark und, das macht Hamburg erst zu einem Hamburg, der Hafen mit dem Handel mit vielen Ländern der Welt. Ihr zitiertes Lied ist schon sehr alt und wohl heute nicht mehr "scenegerecht" um nicht zu sagen "out". Naja, unter dem Einfluß von einigen Litern Bier wird Ihnen hier in Hamburg alles vergeben - auf der Reperbahn in Hamburg wird der Begriff "Sächsischer Stamm" aber wohl eher anders verstanden

|

Mark as:

Currently Being Moderated

35. JackBillingsley,

Jul 15, 2009 4:15 AM in reply to Harm Millaard

Report

Nice article but I have to nitpick a few points:
Rule 1: NEVER partition a disk.
This is incorrect. The partition table it read at boot and never again, unless you make changes to it or force it to (e.g. to rescan disks).
You can even destroy your partition table and everything will run just fine - until you reboot.

Rule 5: Turn off index search and compression. Both will cause severe performance hits if you leave them on.
Correct, but I would just like to point out that the compression feature can be left on - because files aren't compressed by default, only when you ask it to. I know I'm splitting hairs but someone might want to use compression on other (non-relevant) folders/files on their disk and think they can't/shouldn't.

Rule 7: Perform regular defrags on all of your disks.
Defragging is in general overrated but nevertheless yes, it should be done. Personally I recommend using a 3rd party product like Diskeeper which can defrag automatically when needed and/or when usage level is low. Much easier than trying to schedule it and use the built in defragger which is slow and unreliable.

Rule 10: <snip> if you want any protection against data loss, use raid 3/5/6/10/30/50.
I just want to point out that RAID-5 or any other RAID where parity disks are used is likely going to cost you way more performance than all the other mentioned performance points here combined. The read performance is fine but writing is on a different planet, compared to mirroring RAIDS (1, 10, etc). Of course in most cases, RAID-5 is "good enough" but it is very relevant if we're talking about setting up a high-end large storage system, which seems to be the scope here. Also, a good fast RAID-10 makes the points about spreading your data on multiple disks, using separate disks for paging, cache, temp space, etc, redundant - because everything is spread automagically. RAID-5 does not, because the parity disk(s) becomes a bottleneck.

|

Mark as:

Currently Being Moderated

36. Harm Millaard,

Jul 15, 2009 6:06 AM in reply to JackBillingsley

Report

I just want to point out that RAID-5 or any other RAID where parity disks are used is likely going to cost you way more performance than all the other mentioned performance points here combined. The read performance is fine but writing is on a different planet, compared to mirroring RAIDS (1, 10, etc). Of course in most cases, RAID-5 is "good enough" but it is very relevant if we're talking about setting up a high-end large storage system, which seems to be the scope here. Also, a good fast RAID-10 makes the points about spreading your data on multiple disks, using separate disks for paging, cache, temp space, etc, redundant - because everything is spread automagically. RAID-5 does not, because the parity disk(s) becomes a bottleneck.

In fact, raid5 with it's distributed parity also spreads everything across all disks. Even with raid3 with it's dedicated parity drive, the data is spread over the data disks automatically. There is no discernable difference in performance between raid configurations that are mutiples of 10. Raid10, 30 and 50 perform about equal with the same number of disks. The difference is cost. Raid10 is extremely expensive and for most overkill. Example, you need 10 TB of storage. With raid30 or 50 you need 12 disks, with raid10 you need 20 disks plus additional housing and power supply and a larger (more ports, 24 ports instead of 12) controller. Usually when using a somewhat larger array, the bottleneck is the bus. Adding more disks does not give noticeable performance gains after a certain limit.

|

Mark as:

Currently Being Moderated

37. JackBillingsley,

Jul 19, 2009 11:47 AM in reply to Harm Millaard

Report

Harm Millaard wrote:

In fact, raid5 with it's distributed parity also spreads everything across all disks. Even with raid3 with it's dedicated parity drive, the data is spread over the data disks automatically. There is no discernable difference in performance between raid configurations that are mutiples of 10. Raid10, 30 and 50 perform about equal with the same number of disks. The difference is cost. Raid10 is extremely expensive and for most overkill. Example, you need 10 TB of storage. With raid30 or 50 you need 12 disks, with raid10 you need 20 disks plus additional housing and power supply and a larger (more ports, 24 ports instead of 12) controller. Usually when using a somewhat larger array, the bottleneck is the bus. Adding more disks does not give noticeable performance gains after a certain limit.

Yes, the data and parity are striped but this isn't the issue. With raid-5 any write operation means you must read the parity and recalculate it. In raid-10 or any other mirroring raid, you just write it - done. This may not sound like a big deal but is in fact a very real issue in the storage industry. Any redundancy based on parity instead of simply mirroring has this problem. How big of a problem it is depends entirely on your needs and specific setup. Something like raid-50 (which is just two raid-5's striped) will of course be better, but hardly "equal". If you do need good performance (in writes as well), it's much better to just avoid the whole issue by going for raid-10. Yes, it will be more expensive but really, not massively so, and the users we are talking about here likely have photo equipment running into the tens of thousands, and make a living off this data. If you're using SATA, there is no reason to use anything other than raid-10. With SCSI/SAS, the cost difference is much more tangible (e.g. something like 10k vs 15k) but still considering the scope and audience here I consider it a small investment.

The whole other issue is data security. Raid-5 will lose all your data if two disks fail, and even after losing one the chance of having lost or corrupted data is very real. With the size of disks today, you're almost guaranteed to encounter read errors even if a single disk fails.

If you're a single home user or amateur, raid-5 is fine in terms of performance, but I'd personally still avoid it for this reason, plus the cost difference at this low end is very small. For a pro, with a heavy need for large IO operations and multiple simulatenous users, I would never recommend raid-5.

A few links with more details:

http://www.yonahruss.com/2008/11/raid-10-vs-raid-5-performance-cost.ht ml

http://www.miracleas.com/BAARF/RAID5_versus_RAID10.txt

http://www.cyberciti.biz/tips/raid5-vs-raid-10-safety-performance.html

|

Mark as:

Currently Being Moderated

38. Harm Millaard,

Jul 19, 2009 3:31 PM in reply to JackBillingsley

Report

For a pro, with a heavy need for large IO operations and multiple simulatenous users, I would never recommend raid-5.

I quite agree for a real pro, but also a real Pro would not use anything less than a HP EVA Storageworks solution. See here: http://h18000.www1.hp.com/products/storageworks/eva8400/index.html

But then, this Pro would never use PR in the first place. First of all because Adobe is not suitable for multiple simultaneous users. Secondly because a single user does not have large IO operations. He would turn to Autodesk Smoke, Flame, Flint, Lustre, Inferno and the like under Linux.

BTW, have you noticed how many people around here ask questions with less than $ 30K of equipment and how the majority of questions come from people with a camera below the $ 1K boundary? Maybe your perception of PR being "pro" is a bit overrated.

I wonder how many here have an EVA Storageworks 8400 with 324 disks (300 GB/15 K RPM SCSI) for storage?

If you're a single home user or amateur, raid-5 is fine in terms of performance, but I'd personally still avoid it for this reason

You are worried about your 'pro' image when using raid5 instead of raid10? Makes sense if you have an EVA solution. Not so much if you only have 3 or 4 disks, but then a real 'pro' would not limit himself to only 3 or 4 or 20 disks, when he would have to accomodate multiple simultaneous users with heavy IO, he would have at least 96 disks or a multiple of that, of course in a 19" rack.

I don't know whether you realized this was written as some basics for an editing rig, as the title shows, intended for a single user since PR is not intended for network use, so while your remarks have some value in a corporate Linux/Unix based environment, they are completely irrelevant here.

|

Mark as:

Currently Being Moderated

39. Jim Simon,

Jul 19, 2009 7:42 PM in reply to JackBillingsley

Report

If you do need good performance (in writes as well), it's much better to just avoid the whole issue by going for raid-10.

Not sure I agree. In a benchmark test, the Areca ARC-1680 card turned in an average write of 602 MB/s with RAID 3, and 615 MB/s with RAID 10. That's not a significant enough "improvement" in my book to warrant the extra cost.

RAID 30 and 50 were even slower (around 570 MB/s average write).

Further, you'll be reading from a disk as you edit far more often than you'll be writing to it. And here the numbers definitely favor RAID 3, with an average of almost 400 MB/s for RAID 3, and only 257 MB/s for RAID 10.

All this supports a point which I've long expressed. RAID 3 is simply the best choice for media security and speed in a video editing environment.

|

Mark as:

Currently Being Moderated

40. 321 rollem,

Aug 1, 2009 3:50 PM in reply to Harm Millaard

Report

I have a troubling situation. My rig once was a gaming rig with an h20 custom setup. After I configured the loop, the remaining space allowed for 1 HDD, my OS/game/ drive. Now, this machine is loaded with Premiere Pro CS4 and I only have the one drive to work with and I'd like to get a storage drive.

my rig for references; Q9650 8g 4870 Ultra 320 SCSI 300g drive

I know i'm going to need a storage drive for holding my AVCHD files (MTS) so from the get go it looks like thinking outside the case is the case i'm dealing with. Is it a good idea to use NAS for and editing machine & how will it effect my workflow?

Thanks rollem!

EDIT: tried creating paragraphs - this forum program sucks

|

Mark as:

Currently Being Moderated

41. Harm Millaard,

Aug 3, 2009 1:38 PM in reply to 321 rollem

Report

Is it a good idea to use NAS for and editing machine & how will it effect my workflow?

That depends entirely on your NAS configuration. If you have an iSCSI NAS and use teamed NIC's both on the NAS and on your workstation, you may be able to edit effectively, but I have never tried it, so I have no real life experience here. I only use the iSCSI NAS with a single NIC and use it only for backup, not for editing. With a single NIC it is just too slow. Maybe you should look at fibre channel configurations.

|

Mark as:

Currently Being Moderated

42. mikeleuz,

Aug 3, 2009 4:03 PM in reply to Harm Millaard

Report

Hi Harm,

I'm looking to build a system very similar to yours, minus the large raid array.

For storage, I'm looking at:

System/App drive: 300GB WD VelociRaptor drive

Editing Drive: Two 1TB Samsung Spinpoint drives in Raid 0 using onboard Marvell

Pagefile/Scratch/Render: 1TB Samsung Spinpoint drive

(all drives would be backed up to a homeserver)

Is this a decent setup? I'm looking for something faster than 3 single disks, but cheaper than a full RAID solution.

Rest of the hardware would be P6T Workstation, i7 920, good case and ps and cooling, 12GB corsair, gtx260 or 285.

Let me know your thoughts. I need to order tonight.

Thanks!

Mike

|

Mark as:

Currently Being Moderated

43. Harm Millaard,

Aug 3, 2009 9:55 PM in reply to mikeleuz

Report

Looks like a good disk setup. Your media drives on the Marvell will be nearly twice as fast as a single disk with very limited CPU overhead. The video cards are good as well, although the 285 is a bit pricey in comparison to an ATI 4870/4890.

|

Mark as:

Currently Being Moderated

44. mikeleuz,

Aug 4, 2009 5:30 AM in reply to Harm Millaard

Report

Thanks Harm - went ahead and pulled the trigger. I need all the horsepower I can get to handle 5D mkII files!

Thanks again!

Mike

|

Mark as:

Currently Being Moderated

45. Nicolaj T,

Sep 11, 2009 11:39 PM in reply to Harm Millaard

Report

Thank you Harm for very informative guides! One question about storage:

I've just got a new Canon Legira HF200 camera, and I need to build a new PC to edit the AVCHD files. I'm only a hobbyist, so I I'm not going for the high end systems, but I need a system that is reliable and "fast enough" for making editing AVCHD a "positive experience" ...

On my present editing machine (3 yrs old by now) I've got 3x320 GB SATA2 (7200) drives - which I guess are the only components that might be used in my new system. I've also got a WD 1TB e-SATA for backup.

My plans for storage on the new PC is:

1.- 150 GB Velicoraptor for OS/Premiere

2.- Media drive (AVCHD-files, music, pictures etc)

3.- Scrath/pagefiles/???/-drive

4.- A fourth array???

(And it will be a i7 920, 12GB+ RAM, ... system)

The basic question is:

As a hobbyist, how much will I profit of making no. 2 and/or no. 3 (see list above) a RAID-array (0 and/or maybe 1)? I do not think I want to pay a lot to get an expensive RAID-controller. BUT disks are not expensive, so if the performance will increase significantly by using RAID1 (2 or 3 disks), and that can be efficient managed by the MBO - I'll use such a system. I'll use my eSATA drive for backup (and maybe an internal drive for backup as well?).

Of course - I do already have the 3x320 GB SATA2 disks - where would this be most useful, or should I rather invest in som newer and faster (7200) disks?

I guess I'll wait a couple of months before I buy, as I still have some DV-material I have to edit, which work just fine on my previous system. No need to upgrade before I need to -as prices are decreasing and components improve fast! And I guess it is worth waiting for Windows 7 64 bit - am I right?

Thanx

Nicolaj

|

Mark as:

Currently Being Moderated

46. Harm Millaard,

Sep 12, 2009 3:10 AM in reply to Nicolaj T

Report

Nicolaj,

I would initially keep it simple, since the burden of AVCHD editing is on the processor and memory, not on the disk.

1. Velociraptor for OS & Programs

2. 320 G for Pagefile, scratch and final results

3. 320 G for media

4. 320 G for projects

5. 1 TB eSATA for backups or additional media.

Raid1 has no speed advantage over a single disk, it just duplicates the single disk. If you run out of space in the future, you could replace a 320 G disk with a 1 TB or lager disk for you media.

Get a very good cooler for your system, since you want to overclock the i7 for editing AVCHD and get DDR3-1333 memory, not DDR3-1066.

|

Mark as:

Currently Being Moderated

47. Shaluda,

Sep 26, 2009 4:24 PM in reply to Harm Millaard

Report

Harm Millaard wrote:

How do you set up your editing machine in terms of disks for maximum performance and reliability? (SSD's are left out here.)

This is a question that often arises and all too often one sees that initial settings are really suboptimal. These rules are intended to help you decide how to setup your disks to get the best response times. Of course the only disks in an editing machine must be 7200 RPM types or faster. No GREEN disks at all.

What do you mean by "GREEN disks"?

|

Mark as:

Currently Being Moderated

48. Harm Millaard,

Sep 26, 2009 5:26 PM in reply to Shaluda

Report

Green disk are in marketing terms ECO friendly disks that use less energy, because their rotation speed is lower. Like a car, if you do not go over 40 MPH you use less fuel than when driving 70 MPH. Slowing down the slowest component in your computer is bad, like a snail hitting the brakes before cornering.

|

Mark as:

Currently Being Moderated

49. Shaluda,

Sep 26, 2009 5:56 PM in reply to Harm Millaard

Report

Rule 10: If you can easily replace the data in case of disk failure (like rendered files), go ahead and use raid0, but if you want any protection against data loss, use raid 3/5/6/10/30/50. For further protection you can use hot spares, diminishing downtime and performance degradation.

Is there a reference link you could provide as to what all the different levels of RAID do?

|

Mark as:

Currently Being Moderated

50. UlfLaursen,

Sep 26, 2009 8:21 PM in reply to Shaluda

Report

I actually changed a few 'green' discs to black caviar because of your advice, Harm, and I have been really satisfied. Thanks.

/Ulf

|

Mark as:

Currently Being Moderated

51. Nicolaj T,

Sep 26, 2009 11:03 PM in reply to Shaluda

Report

Hi Shaluda,

Here you could find some good information about the different RAID-levels (click on the graphics to make the demo play):

http://www.raid.com/04_01_00.html

Nicolaj

(and thanx Harm for you advice "a couple of posts ago"...)

|

Mark as:

Currently Being Moderated

52. Shaluda,

Sep 27, 2009 2:19 PM in reply to Harm Millaard

Report

Thanks for the explanation Harm. Seems like those "Green" drives would be best suited for backup type functions.

|

Mark as:

Currently Being Moderated

53. Shaluda,

Sep 27, 2009 2:20 PM in reply to Nicolaj T

Report

Thanks for the link Nicolaj; much appreciate it!!

|

Mark as:

Currently Being Moderated

54. 5000cent,

Oct 9, 2009 9:57 AM in reply to Harm Millaard

Report

Thanks for the interesting information. Two questions:

Is a seperate scratch disk for basic editing (AVCHD, little video effects) really needed? Can't you just use a disk for OS/programs and a storage disk for media, project file and preview files?

Scratch disk setup in premiere offers options for captured video and video previews. Assuming captured video means raw source video, this will point to the storage disk. So a seperate scratch disk would only be used for video previews, am I correct?

The WD green 1TB does go to 7200 when needed and therefore seems as fast at other storage disks. Is green really that bad?

@Ulf Larsson: what difference did you notice by going from green to black?

|

Mark as:

Currently Being Moderated

55. Harm Millaard,

Oct 9, 2009 11:34 AM in reply to 5000cent

Report

Is a seperate scratch disk for basic editing (AVCHD, little video effects) really needed? Can't you just use a disk for OS/programs and a storage disk for media, project file and preview files?

It is about the same as having a car with a manual gearbox. Sure you can use that car and drive wherever you like while using only the first and second gear. However, if you also use 3-rd, 4-th and 5-th gear, it will take less time and entail less risks in traffic. Same with the number of disks.

So a seperate scratch disk would only be used for video previews, am I correct?

No, it will be used for the media cache. the indexed, conformed and peak files and possibly for the pagefile, as well as all preview files.

Mechanical disks are the slowest components in a computer. To get optimal results you need to get the last bit of performance from them. Even if what you say is true that the WD green disk can adjust their speed from 5400 to 7200, consider the enormous amount of time needed to get up to speed. That is an eternity in computer terms. Two or three seconds to stabilize at 7200 is extremely slow when disk access is measured in ms. And why? There is no discernable price difference or it can be measured in peanuts.

|

Mark as:

Currently Being Moderated

56. 5000cent,

Oct 9, 2009 4:27 PM in reply to Harm Millaard

Report

thanks for your reply. What confuses me is that premiere does not offer scratch settings for these categories you mentioned: media cache. the indexed, conformed and peak file. Only captured video and preview video.

Assuming a 3 disk setup (OS, scratch, storage), and setting captured video to storage disk (raw footage) and preview video to scratch disk, which disk is used by premiere for what?

|

Mark as:

Currently Being Moderated

57. 5000cent,

Oct 9, 2009 4:26 PM in reply to UlfLaursen

Report

can you please eleborate on the difference you experienced?

|

Mark as:

Currently Being Moderated

58. Harm Millaard,

Oct 9, 2009 5:06 PM in reply to 5000cent

Report

The media cache location can be set in the preferences and contains the indexed, conformed and peak files.

|

Mark as:

Currently Being Moderated

59. Ulf Laursen,

Oct 9, 2009 10:34 PM in reply to Harm Millaard

Report

Hi

I did not have any exact problems with the green ones, I just wanted to be safe based on Harms knowledge. I use the green ones for other purposes like backup and stuff.

/Ulf

|

Mark as:

Currently Being Moderated

60. 5000cent,

Oct 10, 2009 7:24 AM in reply to Harm Millaard

Report

you say you can set media cache in preferences, but I looked and it only mentions captured audio/video and preview audio/video. but I think I understand what you mean: when you set captured video to scratch disk in those preferences, media cache is one of the temporary files that will be stored on the scratch disk, correct?

so for example, I have this setup:

disk 1: OS etc

disk 2: scratch

disk 3: storage with raw footage

I start a new project, I set captured audio/video and preview audio/video to scratch disk 2. And I save the project file itself to disk 3.

After I import video into the project from disk 3, and start editing, all scratch files, including media cache, confirmed audio, preview files, etc will now be stored on scratch disk 2. Is this how it works?

|

Mark as:

Currently Being Moderated

61. Harm Millaard,

Oct 10, 2009 8:06 AM in reply to 5000cent

Report

Correct.

|

Mark as:

Currently Being Moderated

62. 5000cent,

Oct 10, 2009 10:36 AM in reply to Harm Millaard

Report

thanks for pointing out this media cache setting. But I am confused again.

I opened a project on my current PC (just 2 disks, one OS and one data). Scratch disk settings under project/project settings/scratch disks is ''same as project'' for captured video/preview

Than I looked under edit/preferences/media and the location for media cach files is a folder on my C (OS) drive: user/.../AppData/etc.

Than I tried to locate that media cache folder on my PC to see what's in it. There is no AppData/Roaming/Adobe/etc folder on my PC

Than I looked in the project folder on my data disk and next to project file it has these folders:

auto-save

preview files

encoded files

media cache files

this makes my wonder: how did I get a media cache file in my project folder on data disk while the setting in premiere points to my OS disk? And where is that folder mentioned under media cache in premiere? And why didn't they just add media cache to scratch disk settings?

|

Mark as:

Currently Being Moderated

63. Harm Millaard,

Oct 10, 2009 11:20 AM in reply to 5000cent

Report

You ticked the box in the upper right part of my screenshot.

|

Mark as:

Currently Being Moderated

64. 5000cent,

Oct 10, 2009 1:50 PM in reply to Harm Millaard

Report

no, it's unticked (I never touched it). I am starting to believe this edit/preferences/media is simply not working. The media cache setting is overruled by the scratch disk settings.

|

Mark as:

Currently Being Moderated

65. 5000cent,

Oct 11, 2009 6:54 AM in reply to Harm Millaard

Report

Hello Harm,

if you look at scratch disk settings in premiere, what is the ''captured video/audio''' setting actually used for? There is no folder on my harddisk called captured video, while there are folders called media cache and preview files made by Adobe. Maybe captured video = media cache, overruling the media cache setting in preferences, explaining why I do have a media cache folder within my project folder but not on the path mentioned under media cache setting?

|

Mark as:

Currently Being Moderated

66. BrownChiLD03,

Oct 27, 2009 9:57 AM in reply to Harm Millaard

Report

.... i was just wondering.. all these talk about HDD performance.... why did you leave out SSD again? as per my other posts i am planning to build the BEST machine money can buy (whilst still being a somewhat practical) and SSD was my first choice dedicated for OS Drive and Scratch/temp files.. was just wondering why you left it out.. is it a bad idea?

|

Mark as:

Currently Being Moderated

67. Harm Millaard,

Oct 27, 2009 2:10 PM in reply to BrownChiLD03

Report

At the moment I wrote this article, SSD's were too expensive and did not offer a significant speed/performance gain over a good raid setup.

One can still wonder at the current time (october 2009) whether they will offer any significant performance gain with the new controllers and trim function in Win7 over old fashioned hard disks.

For OS, SSD's are approaching the level where you can consider them, a single 160 GB Intel SSD for around € 520 here in comparison to a single 150 GB Velociraptor for around € 140.

For editing the capacity is still a major bottleneck. I prefer a 12 TB raid30 with 800+ MB/s transfer rate for around € 1600 (including a top notch raid controller) over a 640 GB SSD raid0 with around 750 MB/s transfer rate for around € 2080.

|

Mark as:

Currently Being Moderated

68. tennisboy72,

Oct 29, 2009 5:27 AM in reply to Harm Millaard

Report

Harm,

First of all, I'd like to congratulate you on your excellent articles about your hardware configurations. I've just ordered a new pc and I'd have a question for you. I would like to set up a system with one HDD for my OS and 4 HHD's in raid 0 config for my main storage with a hardware raid card. But mostly on the net when you reed about raid, it's always a "total" raid config (with all the HDD in raid & OS also on the raid) and mosstly a software raid.

Also some scary story's about suddenly having a pc that's not bootable anymore...

So my question is how about am I to go to work ? I tought of the following scenario :

1) set up pc with 1 HDD and install win7 ultimate 64bit on it.

2) open up pc, put hardware raid controller card & 4 HDD in it. Also connect the 4 HDD to the hardware raid card.

then I'm not so sure, do I have to go into the BIOS at next startup to set up the 4 HDD in raid (or is this only needed for software raid ?)

Or do I only have to config the 4 HDD with the setup software that comes with the raid card ?

Sincerely,

Frank from Belgium

|

Mark as:

Currently Being Moderated

69. Harm Millaard,

Oct 29, 2009 9:02 AM in reply to tennisboy72

Report

Frank,

First a side remark about Win7 Pro versus Ultimate. Pro is cheaper and has a smaller footprint, do you need the BitLocker and Media capabilities? If not I suggest Pro.

Another side remark is that Intel withdrew the trim function update because there were numerous complaints about the hardware upgrade eating the SSD's. This will only be temporary, but be warned not to try the hardwate updates at this moment.

You can install the SSD and install your OS and programs. It is irrelevant when you insert your raid controller into the system, because the setup of the raid will be done during boot, before Windows starts. I would seriously look at an Areca ARC-1680iX-8 card, not cheap but the best money can buy. The iX-12 model has the added advantage of allowing more cache memory to be installed, apart from the extra ports.

I would not advise a 4 disk raid0. If one disk fails, you lose all your data. I would prefer a raid3 (with Areca) or raid5 with nearly all cards and build in some safety or add another disk to the array if performance is so important to you. Just the other day I had one disk fail and had to exchange it under warranty. Inserted the new disk, built a hot-spare and expanded my array, which started rebuilding immediately and I could easily continue work without losing a single byte of data. It was degraded for some time, but hardly noticeable in performance.

|

Mark as:

Hardware Forum

More Like This

Incoming Links

Bookmarked By (3)

Hardware Forum

Storage rules for an editing rig. Some basics.

May 3, 2009 12:19 PM

More Like This

Incoming Links

Bookmarked By (3)