Massive Disk Speed Improvement Plan

Actually, with 2 TB on tap (which is what I have now with the spinning array), Windows quite likes to have its OS, apps, and even most data files right on C:.  This is probably a throwback to ancient times, but by golly WIndows and its apps always seem to work best in all the default locations.  This system is rock solid stable.  And keep in mind I use it for more than just Photoshop...  I develop with it, run my business with it, and many other miscellaneous activities, some of which get pretty intensive.

Of course once I have C: set up as the array I'll make all the spinning drives into a killer D: data drive as well, for stuff that doen't need microsecond access.  The best part is that I can continue working exactly as I am now, with no need to rearrange anything, and with plenty of space to spare all around.

I'm looking forward to experimenting with whether Photoshop scratch will be better on C: or D:. 

Here's how my (spinning SATA2) drives benchmark now - this is my baseline.  As you can see, drive C: is no slouch already...

C:, 2 x 1TB WD RE4 in RAID 0:

D:, single, non-RAID Hitachi:

I'm working now on how best to benchmark Photoshop's actual performance as it "goes virtual".

-Noel

Water?  Think liquid helium, my friend. 

No, seriously, I already thought the power and heat loads through...  These things draw less than half of what a spinning drive draws - 3 watts when actively being read/written, 1.65 watts just sitting there.  Cool as a cucumber!

I've developed a benchmark based on an action doing things to a 3 GB image file that drives Photoshop to get well into the scratch file - to the tune of 30 GB.  I've already learned that I've had the scratch drive pointed in the wrong place all along and that it runs better if pointed at my drive C: (spinning disk) array than my separate D: drive now.  This action spends 3 minutes kicking hell out of the scratch drive.  I imagine it will complete a little more quickly with the SSDs in place.  Now to wait for the UPS man.  Is he here yet?  No.  Is he here yet?  No.  Is he here yet?  No.  Is he here yet?  No...

I have learned that Photoshop CS6 is about 10% slower than CS5 doing the same things on a very large file, which is kind of a surprise.

-Noel

Jealousy noted.  Thank you. 

To be fair, this workstation is at the heart of my software development business, so it's not all play.  I run a lot of development tools, including virtual machines for testing, that will be GREATLY enhanced by not having to wait on disks.  If at some point it helps me get a product released a few days sooner the upgrade will have paid for itself.

And yes, in hindsight it does seem obvious that the RAID C: drive would be better.  But the lore has always been to keep one's Photoshop scratch file on a separate drive from the system, and I've followed that advice without really thinking about it or questioning it.  It's high time to question everything. 

-Noel

Lundberg02 wrote:

 

Why do you use RAID 0? I would think you would use RAID 1.

RAID 1 buys you redundancy, not an increase in performance.

-Noel

I keep good backups, and as I mentioned I plan to test it pretty well for a while as a data drive.

They've crossed the threshold to "reliable enough" of late, and these OCZ models, with 2 million hour MTBF and 3 year warranty, are reviewed as being pretty solid.

-Noel

Well, here's what I have so far for you, Chris.  I may have overestimated that 10% a little.

1.  I've set the performance settings the same in CS5 and CS6.  I have 100 history states in each.

2.  I have a very large image with multiple layers.

3.  The image I run the action on is a fairly large multi-layer 16 bits/channel image with some adjustment layers.

4.  The action duplicates layers in groups, and makes a bunch of pixel editing changes on an image layer so as to create lots of history data.

One thing I've noticed is that I probably have the Photoshop RAM limit a bit high, because while watching the Resource Monitor I noticed some Windows swap file activity, as well as a ton of scratch file activity.

Results:

Drag uncompressed 3.1 GB PSB file and drop it onto the Photoshop icon (Photoshop is not running)

23.0 seconds:  File not cached, start of Ps CS5 x64 plus file open

14.8 seconds:  Second time, file cached, start of Ps CS5 x64 plus file open

237 seconds:  Run of the action in Ps CS5 x64, tons of scratch file activity.  Scratch file grew to 30 GB during this run.

--

24.8 seconds: File not cached, start of Ps CS6 x64 plus file open

15.2 seconds: Second time, file cached, start of Ps CS6 x64 plus file open

183 seconds: Run of the action in Ps CS6 x64, tons of scratch file activity. Scratch file grew to 30 GB during this run.

So where Ps CS6 is ever so slightly slower to open the same file, it's a fair bit faster to do the same operations on the file in which a lot of scratch file I/O is required.

Of course I'll repeat these same tests when I get the SSD RAID set up. 

-Noel

I have begun to execute my plan.

To date I have purchased four and installed two OCZ Vertex 3 480GB SSDs, and have been doing methodical testing.

Findings so far:

In actual use, the SSDs effect a HUGE improvement in responsiveness of whatever's being done on them, as they have instantaneous seek time.  This was apparent even with one SSD as a simple drive.  Putting two of them into a RAID0 array hosted by my motherboard RAID controller didn't change the responsiveness significantly, though it doubled the maximum throughput so operating on large files is much faster.

The benchmarks show a modest improvement over my spinning drive array in small transfer speed, and a large improvement in max transfer rates.  Remember that all these speeds are still limited by the motherboard SATA2 RAID controller I am using while awaiting the high performance Highpoint controller that's on order.  Each SSD is essentially saturating one SATA2 connection.

My Photoshop action test on large file, with heavy scratch drive activity, faired as follows, using my single spinning drive, my RAID0 spinning array, and the new RAID0 SSD (two drive) array.

266 seconds - Photoshop scratch on D: (single Hitachi spinning drive).

189 seconds - Photoshop scratch on C: (RAID0, dual spinning Western Digital RE4 drives).

171 seconds - Photoshop scratch on E: (RAID0, dual Vertex 3 SSD drives).

Most notably, I have run several different virtual machines entirely off the SSD array, and comparing that experience with the same virtual machines running off the spinning drives is night and day - Windows boots up in 1/4th the time, everything just feels really snappy, and notably the systems don't load up as quickly.  If one starts some intensive disk activities, such as a search of the entire drive for something, then starts something else, the impact is barely noticeable.

Some interesting numbers:

92 seconds - Cold boot Windows 8 VM to the lock screen on drive D: (single Hitachi spinning drive).

32 seconds - Cold boot Windows 8 VM to the lock screen on drive E: ((RAID0, dual Vertex 3 SSD drives).

The high performance SATA3 RAID card that promises to provide a significant boost in both IOPS and individual drive speed is due to be delivered by UPS later today.  Watch this space!

-Noel

No, I have installed Ps CS6 on my host system.  It ran well enough (i.e. perfectly) in a VM for a few weeks so I went ahead and installed it on my host hardware so as to test it with my video card (all is well).  That's the configuration I used to do the above testing.

Notably it takes about half the whole benchmark time of roughly 2 to 4 minutes just to start accessing the scratch file, as quite a while is spent filling up the RAM.  And from what I can see in the Performance monitor it never goes quite to the full transfer rate to the disk - just to about 170 MB/sec.

-Noel

Wet blanket!

Hey, some things are just worth spending money on if for no other reason to learn about them.    I now know MUCH more about this technology than I did before.

In all seriousness, I use this workstation 16 hours a day, for everything I do.  Having it be more responsive will improve everything I do.  But no, the speed improvement, while clearly felt in most aspects of using the computer, is probably intangible enough that I would not say it's a cost-effective move to buy a bunch of SSDs just yet.  Wait until the prices come down a bit more.

These disks will go into my next workstation as well, no doubt.  Looked at another way, when I buy my next workstation, I'd have to figure out what disks to put in it then.  Now I have that part already covered.

By the way, I've been tweaking it and today I hit the 1 gigabyte per second mark using the SATA3 controller.  And I haven't even got all the disks in the array yet (two are still on a UPS truck somewhere).

-Noel

Hey!  I got out just last month.  That's enough for one decade. 

-Noel

What's the payback for learning new technology?  That I'll be able to create products to take advantage of it sooner than others who don't know about it?  Certainly there's value there.

As far as payback time to see improved performance, it's immediate (after the installation, of course).  How long will it take to pay back the cost because I have a product done sooner?  If somewhere down the line I have a product ready and on the market a week or two sooner, it's paid for itself.  When will that occur?  I don't know.  Will it occur?  I imagine so, yes.  This workstation is at the center of the engineering work I do.

I'm the CEO.  I get to buy things from time to time without having to justify the expense to someone else. 

And besides the value of learning new tech, and justifying the cost, there are intangible factors...  You picked one car over another last time you bought one.  Perhaps you bought a sports car - let's say you did.  Did you have to justify to someone how much more money you'd make by being able to drive your sports car to work rather than a run-of-the-mill sedan?  I'd wager it costs you MORE to drive that sports car, yet you still bought it... 

-Noel

Since you're wanting to focus on the tangible, Lundberg, here's one early result (bearing in mind I'm not done yet and I'm only testing the SSD array as a data drive now)...

One of the things I regularly do is boot up multiple different virtual machines, to facilitate testing my products in multiple environments.

It used to take 2 to 3 minutes to boot up a VM.  When hosted on the (2 drive) SSD array they boot up in under 30 seconds.  Software installation took a minute, now it takes 30 seconds.  Shutdown took a minute.  Now it takes 9 seconds.

Conservatively speaking, the SSD array saves more than 15 minutes to go through one testing cycle using 6 VMs (XP, Vista, Windows 7 32 bit, Windows 7 64 bit, Windows 8 32 bit, Windows 8 64 bit).  Maybe even more because I will be able to boot up several at once, which was impractical before.

Software builds will be quicker, review comparisons will be quicker, subversion tag creation will be quicker...  And of course Photoshop operations on big images are quicker.

It all adds up.  There are good reasons for having a fast computer over a slower one.

-Noel

Heh, my Windows 8 VM just downloaded 18 updates, installed them, and rebooted, and start to finish this took under a minute.

-Noel

Not sure who might be following this thread, but I have executed the final phase of this plan, restoring a system backup from my spinning drive array onto the new 4 drive SSD array.

All went off without a hitch, I have my same system configuration including all apps and everything just as it was, except everything is now MUCH faster.

The 4 drive array achieves a staggering 1.74 gigabytes/second sustained throughput rate.

Windows 7 WEI score is 7.9 for the Primary hard disk category.

Windows boots up quickly, everything starts immediately, nothing bogs the system down, and just overall everything feels very fluid and snappy.  And there is no seeking noise from the drives.

Regarding what this has done for Photoshop...  I've only tested on Photoshop CS6 beta so far today, but everything is incrementally improved.  Startup time is faster, things seem more smooth and fluid while editing overall, and a benchmark I created using an action to run a lot of image adjustment operations on a big, multi-layer image ran this long to completion:

When the file is opened from (and the Photoshop scratch file is on) a single spinning disk: 

4 minutes 26 seconds (266 seconds)

When the file is opened from (and the scratch file was is on) a fast array of spinning drives: 

3 minutes 45 seconds (225 seconds)

When the entire system is run from the SSD array: 

2 minutes 31 seconds (151 seconds)

During the action, because so many steps are performed on the big file, Photoshop writes a 30+ gigabyte scratch file on the scratch drive.

Summary

Clearly the very fast disk access markedly improves Photoshop's speed when it uses scratch space. 

Plus copying big image files around is virtually instantaneous. 

I don't use Bridge myself, but I have noticed that all the image thumbnails (via FastPictureViewer Codec Pack) just show up immediately in Explorer windows and Photoshop File Open/Save dialogs.  We can only assume this kind of drive speed would really make Bridge blaze through its operations as well.

Following my footsteps would be expensive, but it can really work.

-Noel

Thanks guys.

Chris Cox wrote:

 

Impressive.

 

I'd be curious to see how it affects launch time on CS6.

Not quite as much as I thought it would, though it's faster in just the right ways...  It comes up to the main window painted state maybe a second faster out of about about 5 or so, which seemed always to take a second or two longer than Photoshop CS5 (still true).

I just did some timings with my stopwatch, starting when opening an icon on the desktop and ending with the Photoshop main window painted.

Photoshop CS5 x64: 3+ seconds (averaging around 3.2).

Photoshop CS6 x64: 4+ seconds (averaging around 4.5).

More importantly, however, there was always a bit of wait time - a few seconds - after the Photoshop CS6 UI was displayed before the menus were available that's been reduced below the threshold of pain for me.  Now I see that however long it takes me to get my mouse lined up with the File menu is just about the time it takes to be ready for me to click it.  When it responds to my click the first time the cost of this array becomes a little more worth it. 

pf22 wrote:

 

I'd certainly be interested to see how long it takes you to burn through all that lovely expensive NAND!

Given the Sandforce drive controller's "wear leveling" facilities, and the gargantuan amount of space these drives have to go through, I did some back of the napkin calculations that showed it should be a non-issue (10+ years).  Probably by the time any of the drives need replacing I'll be looking at the newest technology at the time, which will no doubt far outpace these drives for both speed an size and be cheaper.  The drives I chose have a 2 million hour MTBF as well, so the chance of failure being multiplied by 4 drives in the array is still miniscule.

The drives do report, via SMART, a projected lifespan.  I don't know how likely it is to be accurate, but so far in a week of hard benchmarking and testing I haven't gotten any of them to budge off 100%.

-Noel

Lundberg02 wrote:

 

When SSD is 25 cents a gigabyte, call me. I'll be happy with eSATA until then.

Stick in the mud.  Meanwhile we early adopters will be enjoying system performance you can only dream about...  You know you want it... 

Good thing at least some of us step up to help finance new technology development!

-Noel

Lundberg02 wrote:

 

if only the web was faster than 25 mbps actual.

Have you been talking to Al Gore again? 

-Noel