Hard drive configuration with NVME/SSDs

[RockyIV]

I have a system with a 1TB NVME M.2 Samsung Evo 960 and a 500GB SSD Samsung Evo 850. My understanding is that the best configuration would be:

1TB NVME M.2: OS, Apps, Disk Cache

500GB SSD: Project Files / Source Files-footage-images-etc.

I typically archive my projects to a local NAS + Cloud storage, so 500GB works fine as a project file folder. I don't mind swapping them though and having my disk cache on the SSD if the performance difference isn't that great. My machine also has a second NVME, and I do have the option of the following...

1TB NVME M.2: OS, Apps

500GB SSD: Project Files / Source Files

500GB NVME M.2 (partition from another drive): Disk Cache

I'd rather not partition my other NVME drive, but I'm willing to if there is enough of a performance difference by having 3 drives. Which configuration would be the fastest for After Effects? Also, with either configuration, should I be rendering to the same folder as the project file or the disk cache for optimal performance? Or does that even matter?

Thank you for any advice. Sorry if I missed this topic before. I haven't found many threads regarding system performance of NVME mixed with SSDs.

[Roland Kahlenberg]

The fastest HD should be used to store active media.

[Szalam]

I disagree with Roland. The cache should be on the fastest drive per most of the testing I've seen.

[Roland Kahlenberg]

Looks like a one-arm battle is asunder. 🙂
Which test(s) are you alluding to?

Here's my take. Cached files from a single timeline are in essence a single layer when called for a preview. One only needs a drive that is quick enough to play back a single stream of video at the required video rez for frames cached to the HD. We only need a HD which is fast enough to feed data into RAM without the user having to wait. On a 550Mbps SSD, this should be instantaneous. Additionally, AE is intelligent enough to calculate when to initiate playback based on the transfer of the cached (to HD) frames such that the preview is not consistent, ie does not stop midway while cached (to HD) frames continue to be transferred to RAM.

If frames have not yet been cached to the HD then it's a different story. Here, we want a super fast drive that loads as many layers as the timeline has, within the Preview Range. So, we're talking about many streams here. Hence, the fastest drive for media is ideal.

[Martin_Ritter]

I would put OS and Software on the SSD. It is fast enough and you are not spending the whole day opening software or rebooting the computer. So you hardly benefit from the speed of an M.2.

I always put the cache on it's one drive, M.2 in my case currently.

If readings from Windows are correct, AE only transfer data with less than 100MB/s to the cache (I'm doing a lot of motion graphics, no idea how this change when making VFX), so there is not really potential to speed things up. M.2 however has the shortest response times and can handle the most I/O operations, which is for sure useful for the cache drive.

Use a 2nd M.2 for project and footage if you like. I'm working with a NAS, too. Connection is 1GBit/s, NAS can output 400MByte/s and this only bottlenecks when working with 4K video, which I hardly do.

When working with motion graphics, character animations and stuff like that, I usual spend more time hitting the RAM border or waiting for the preview rendering. Loading assets, even with many layers, is not really an issue or noticeable.

[Peru Bob]

I'm moving this to the Hardware forum where the hardware experts hang out.

[Kyle Hamrick]

This article that just came out might be helpful:
https://www.schoolofmotion.com/tutorials/after-effects-computer?ref_id=ae-comp-tw

[Szalam]

I was just about to share that!

[Roland Kahlenberg]

Kyle, there's actually a lack of clarity where cache drives is concerned.

"Try to use an NVMe for the disc cache."

This is not good advice. Puget's own test show the difference between a SATA SSD and an NVMe to be negligible for a Cache Drive. When we consider the cost differential between these drives, it's not good advice.

I'll also go further to add that the calculations provided by puget in their tests here https://www.pugetsystems.com/labs/articles/Adobe-After-Effects-CC-2017-Disk-Cache-Performance-Analys...
does not tell the whole story.

I'm pretty certain the times they've calculated are based on loading of all the cached frames from disk to RAM. However, AE doesn't have to wait for all frames to be transferred from disk to RAM before it begins playback of the preview. Hence, we should only be looking at how fast AE begins playback, to compare between different storage devices. The length of the Preview Range should also be considered since I'm pretty sure AE does some calculation to decide when to actually begin playing frames after receiving a call for a preview - the longer the preview range, the more time AE will need to calculate and decide when to begin playback. However, even this is likely to be negligible.

Here's a screengrab of Puget's tests (link is above)

[Kyle Hamrick]

Some of this is definitely getting into those areas where you MAY see slight benefits, assuming everything else is configured properly. Whether it's worth the extra cost is a different question. The form factor of NVMe is a nice benefit too, and prevents you from having to run so many cables around, etc.

I'm currently running both my project files and cache off a single NVMe, on the theory that they're fast enough that it shouldn't really matter if it's being accessed twice at once anyway.  ¯\_(ツ)_/¯

[Roland Kahlenberg]

But conventional HDs (spinners and SSDs) can only read or write at any one time. And it can only perform one read/write at a time. I think you'll get some productivity improvement even with an external USB SSD with the proper connections which should allow for approx 450-550 MBps.

I just realized the real (noticeable) difference in using an NVMe over a SATA SSD for cache is when emptying the cache - this is noticeable and affects workflow.

BTW, if you're doing PPro work, then a dual NVMe is ideal.

[MarkusTheArtist]

Any help on dual NVMe setups...thats a new concept for me and I am currently starting a build for a new Piston's Production/VFX machine

[Roland Kahlenberg]

Media Files on the fastest drive you have. Cache drive on a drive sufficiently quick to play a single stream of video in real time because that's what cached media essentially is when it's sitting on your timeline. You may want to test putting your cache on the second NVMe drive to see if there is any difference to to the cache being on say an external SSD rated at 450MBps, read/write. Writing and reading cache data is essentially writing and reading a single stream of video - 450MBps read/write should suffice AFAIK; with very little differential when compared to an NVMe that writes/reads in excess of 1 GBps.

One area which has received little mention is the time it takes to purge AE's cache. Since purging cache is a frequent task, the time it takes is multiplied on a per day basis and then over a week and then over a year. The productivity gains may be quite substantial.

[Neofilm]

I think I want to spare my NVME's the abuse of cache so I'll probably keep that on a new SSD.

Thanks for such a thought through response

[Neofilm]

When you say dual NVME for PPRO is ideal, what do you mean exactly? I have the capability for that but I'm not doing that currently. Right now I have potential for 3 with 3 m.2 slots. I only have two though. One is for OS/Programs and I'd like to keep it that way. The other is currently for 360 tours but it may be overkill. I'm hoping to get the third soon for PPRO, saving up for it...

I also have 3 SSD's. I've been using one of those SSD's for project and media files and using another for cache. Everything has been great though, this is a new build. I have however been having some insane problems after about an hour of editing (here's the thread if you're curious: Re: Warning, error, frame substitution??? )

[RockyIV]

Thanks so much for the information everyone. Didn't expect so many in depth responses.

Most of my storage information is actually form a Puget article here: https://www.pugetsystems.com/recommended/Recommended-Systems-for-Adobe-After-Effects-CC-144/Hardware...

Basically, the article says...

SSD for OS/Apps

SSD/NVME for Project Files

NVME/SSD for Disk Cache, in which they write for a disk cache drive "Upgrading to an even faster NVMe drive can give some performance gains, most notably allowing the disk cache to be written on average around 20% faster."

I can't for the life of me find this post, but I had read an Adobe rep's response to a HDD configuration regarding two drives. They had said it's okay to use OS/Apps/Disk Cache on a fast disk 1, with project files on disk 2. The reasoning was that once the OS  is running and apps are opened, not much reading/writing is going on with that drive. A fast third drive wouldn't have much benefit over using just two drives. This is why I was wondering if anyone had experience with just using two drives. If there is little benefit to using a third drive, I'd rather just stick with two.

I am just an After Effects user. I jump into all the other programs when needed (PS, ILL, PREM, C4D, etc) but 90% of my time is in AE.

[Martin_Ritter]

I prefer to put the cache on it's own drive - as said.

The reason is, that the OS drive does write or read data, even if this is not much. Also, OS swaps the RAM to this disk and than maybe there is a security scanner, which randomly checks the files. All those operations - even if not much in summery - interfering the cache read/write process. And if I can, I want to make this process as ideal as possible.

Also, cache drive get's worn out faster. It will took a long time until it dies, but I want to assure that I only have to replace an non critical SSD/NVME when it happens.

In addition, there is the space aspect. When I can have 500GB or 1TB for cache, why share it with OS and software? Those drives are not that expensive, especially SSDs.

[RockyIV]

The OS and apps don't take up too much space, so sharing them with the disc cache on a 1TB NVME would be nice. Especially if there's very little performance hit. SSDs might not be expensive, but they do take up SATA ports. 2 NVME drives use up 3 SATA ports alone. Using a single NVME for OS/Apps/disk cache would save ports, money, and is possibly faster than using a separate SSD for disk cache (the big mystery).

Good point on wear. I'd definitely be curious if it poses an issue with an NVME Samsung Pro. Especially if it's being shared with the OS.

Once I have a break from work and can put my new system together I'll definitely try out these configurations and see if there's any noticeable difference.

[Martin_Ritter]

About the worn-out aspect: I used to have an Cache-SSD (Samsung 860 Pro) and after I checked the TBW after 5 years of constantly use, it was not close to anything to worry about. I don't remember the number and I can't look it up right now, but it was something like only 10 till 25% of overall TBW used.

Latest test shows, that official TBW numbers are below of that those drives can actual handle, too.

It's an aspect, but not a very huge one, I admit. However, following Murphy's Law, I rather separate critical and non-critical parts of the computer physically.

[Roland Kahlenberg]

What do you find mysterious with AE's Disk Cache?

[Neofilm]

Hello all, I would love some advice in respect to this discussion and my new build.

I just built a new system and currently it only has two m.2 drives.

D: Drive is a new 1 TB NVME Samsung Pro. I'm going to use this for video projects and video assets (do you guys think it's good to put the project and assets on the same drive?)

C: Drive, the other m.2, is an older 500 Gb NVME Samsung Pro. I pulled it from an older computer that was using it on a PCIe card. It's for my OS. Is it a good idea to use the OS drive for video project cache?

I have room for another m.2 but I'll probably use this for 360 photo work.

I have room for a few SSD drives and/or mechanical drives. I'll probably get one SSD for active project backups and one more drive for archives.

It would be awesome if some true, honest, objective authority could give us the best option for a typical computer setup. I imagine most are like me, just trying to setup a few drives, no RAID, just one drive for OS, one for projects, do I need a third drive for cache and if so what's the best?

Lastly, recently someone suggested I consider using RAM for Cache. I understood him to suggest allocating a certain amount of RAM for cache because it would be the fastest way for cache to work and it's completely nonvolatile. What are your thoughts on using RAM for cache?

If you're wondering, my system is:

i9 9900K, 64 Gb 3600 mhz RAM, 2080 ti video card, Gigabyte Aorus Master mobo.

[Martin_Ritter]

For the objective authority, follow the Puget-Systems recommendation - link is above in one of the posts.

From my experience, project files and assets on one drive are just fine.

When it comes to the OS and Cache combo, there is no clear advice. Read about the pros and cons in the posts above.

Creating a RAM-Drive for cache is not very useful in this case. First, AE will need all of the memory and can even fail during rendering, if there is to less RAM left. Second, transfer speeds to cache are kind of low, so you don't benefit from RAM drive speeds (see the post above, there was a screenshot with cache typical speeds). SSD is fine, NVMe more then enough.

*Martin

[Roland Kahlenberg]

Media Files on the fastest drive you have. Cache drive on a drive sufficiently quick to play a single stream of video in real time because that's what cached media essentially is when it's sitting on your timeline. Hence, a drive that can transfer at a rate of 450-MBps will be sufficient for up to 4K video - I think, if my calculations are correct.