• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
5

Using External HDD's for Video Editing

LEGEND ,
Jan 28, 2011 Jan 28, 2011

Copy link to clipboard

Copied

External HDD’s (Hard Disk Drives) are everywhere, and quite popular. These allow one to increase the storage capacity of a system greatly, quickly and easily, for not that much money. We now have many sizes, and many connection types. Some connection types are better than others, if one is doing more than off-loading data to archive it, and perhaps free up internal HDD space. With the right connection type, one can also edit to/from the external HDD’s. Again, some connection types are better than others. Note: for editing Video, the physical HDD’s need to be at least 7200 RPM.

For simple archiving, any connection type will work well. In that process, files are just written to the external HDD, and when needed, read from the external HDD. The problem comes in, when one tries to edit to/from the external HDD. Most current computers are faster than the slower connection types can manage. The OS is firing commands for reads/writes too fast. The OS expects rapid response, and can overwhelm the external HDD. When one adds in an NLE (Non Linear Editor) program, there is a great deal of HDD activity, and this requires many reads and writes, and requires them quickly. The files are very large too. Suddenly, the I/O (Input/Output) sub-system is overwhelmed, but the requests keep coming.

Here is a real-world look at the data transfer of some common connection types:

Average transfer rates in MB/s for different interfaces:

  • USB2:    20 - 25, depending on other USB devices sharing the same bandwidth
  • FW400:  30 - 35
  • FW800:  50 - 60
  • USB3:    65 - 80, depending on other USB devices sharing the same bandwidth
  • eSATA: 100 -140
  • SATA:   100 - 140

Raid0:    0.9 x N disks over a single disk

Raid3/5: 0.8 x (N-1) disks over a single disk for read, 0.6 x for write. ICHR10 figures are a bit lower than hardware controllers.

Note: “FW” refers to FireWire, which might also be referred to as IEEE-1394a (FW-400) or iLink, and as IEEE-1394b (FW-800). The connectors are different, as are the cables. Also, one needs the appropriate controllers. Most computers have FW-400/IEEE-1394a/iLink controllers on the MoBo and connectors available for the cables. For FW-800/IEEE-1394b, one usually needs to add a controller, either on a PCI card, or via PCMCIA, or ExpressCard. More on controllers later.

As one can see, the USB 2.0 connection is slower than the rest. Problems can arise, when trying to edit to/from such an external HDD. They can be plagued with read/write errors, as they cannot maintain the data transfer, required by the OS and by the NLE program. When the allowed time has expired, based on the OS’s expectation of data transfer, it stops the process, and issues a read/write error. In a worst-case scenario, the OS will have routed the write data into a buffer, waiting for the external HDD to catch up. These buffers are finite in their capacity to store the data, while the OS is waiting. It does not take too long, before the dreaded “Delayed write failure” rears up. This can lead to a catastrophic failure of the external HDD, as the data is only partially written, and the File Allocation Tables are not completely written, causing the loss of ALL data on the external HDD. At that point, all data on the external HDD is lost. It might be possible to do a recovery with a program, like Stellar-Phoenix, but it is not cheap, not is it fail-safe. Depending on the value of the files lost, one might hire a data recovery service, but these are anything but cheap. Probably best to just do a complete low-level Format of the external HDD and then do an OS Format.

Also, while speaking of Format, most external HDD’s come from the factory, Formatted to FAT-32, an older Format type. Besides being slower, FAT-32 has a max. file size of about 4GB, and AV files are often much larger. One needs to CONVERT the Format to NTFS for the PC, or to the Mac OS’s Format, to allow for the larger files, and to increase the speed a bit. On the Mac/PC front, a PC can read/write to FAT-32, or NTSF. However, a Mac can ONLY read/write to a PC external HDD, Formatted to FAT-32. If one is migrating the HDD’s between Mac’s and PC’s, the external needs to remain in FAT-32 with its limitations. Just one consideration, when setting up external HDD’s for use with an NLE program.

For editing Video, I have found that the connection types should be considered this way:

  • USB 2.0 - archiving for storage only
  • FW-400 - archiving for storage, and light editing to/from (just very slow)
  • FW-800 - archiving for storage, and regular editing to/from (fairly fast)
  • eSATA - archiving for storage, and regular editing to/from (about as fast as an internal SATA)
  • I have not tested USB 3.0 in editing situations, so cannot comment on their acceptability.

Obviously, a SATA internal is ideal for all tasks.

Now, back to controllers. The real-world throughput of an external can be diminished due to the controller, and the number of chips available. Many MoBo’s have 1 - 2 USB 2.0 chips, and 1 - 2 FW-400 chips. One needs to check the number of chips provided. One might have multiple connectors, either on the computer’s case, or direct from the MoBo. These connectors might go to multiple controller chips, or might all feed into one. The same can be said for add-on cards, PCMCIA cards, or ExpressCards. The number of connectors does not guarantee the number of chips.

The more externals, and peripherals that are connected to the controller chip, the lower the data transfer rates. This holds for hubs especially. One might have various externals and peripherals plugged into various connectors, but everything being fed through a single controller chip. Add to that, the ability to daisy-chain multiple externals through the In/Out connectors on most external HDD’s, and overall performance falls even more. USB 2.0 and FW-400 suffer most from these multiple externals limitations, but FW-800 can too, just not to the same degree.

When one is using an external to edit to/from, there is another consideration. When peripherals with mass storage capabilities are plugged into a computer, a drive letter is assigned by the OS. Most NLE programs, like PrE/PrPro rely on the drive letter to link to the Assets. They do this via an absolute path, including that drive letter. Depending on how many other peripherals are plugged in, and the order that they are plugged in, that drive letter assignment might differ, session to session. When using external HDD’s, one should go into the OS’s Disk Management console and assign a fixed drive letter for that individual external HDD. This would hold for however many one is likely to use, and each should have a unique drive letter. I suggest marking each external with the drive letter that the user assigns to it, say Z:\. Then, whenever Z:\ is plugged in, it will always be seen as Z:\. This way, the NLE can keep up with where the Assets are located, starting with the drive letter. If one is migrating Projects between computers, they will repeat this exact process in the OS of each computer.  Note: when doing the migration, ALL Assets, Scratch Disks, and the Project file, MUST be included on that external, or one will spend a good deal of time tracking down the Assets.

External HDD’s are great devices, but some considerations need to taken into account, before one just plugs them in, and begins editing.

Hunt

TOPICS
Tips and tricks

Views

96.0K

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Feb 06, 2012 Feb 06, 2012

Copy link to clipboard

Copied

Video editing is actually one of the most like apps in many iphone these days. But recently many longtime consumers of Apple's Final Cut Pro non-linear video editing software were upset when the business upgraded to Final Cut Pro X last summer. The software has been a staple for video editors, pro and novice, since 1998. But the new edition is incompatible with files from old versions of the software, and the interface has been entirely redesigned. Apple released upgrade 10.0.3 Tues in an attempt to correct some of the troubles. Resource for this article: New update for FCP X addresses user complaints.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Feb 06, 2012 Feb 06, 2012

Copy link to clipboard

Copied

Not sure how this applies to editing with external HDD's, unless one can hook one to their iPhone, but yes, many have jumped ship from FCP, and are embracing PrPro CS 5.5. While some find a learning curve, such as PrPro editing most format natively, rather than by proxy files, and some operations between the two NLE programs not being exactly the same, most are greatly enjoying PrPro CS 5.5, on a proper computer.

Thanks,

Hunt

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guest
Jul 05, 2012 Jul 05, 2012

Copy link to clipboard

Copied

I'm undecided on how to set up a raid 0 not sure if internal hard drives would be best or docked external hard drives? which would work best in video editing.. I'm looking for speed and safety which from my understanding it's one or thee other?... I have a Mac pro 2.8 quad-core intel xeon 10.6.8 with 7gb of ram I do all editing with adobe.

I don't have a huge budget so i'm looking to know whats necessary for smooth work flow.

Thanx in advance!

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Jul 06, 2012 Jul 06, 2012

Copy link to clipboard

Copied

This ARTICLE gives some tips on setting up a RAID. Also, there are many great threads on RAID configuration and control, on the Premiere Hardware Forum. Be sure to see any reply, or article, by Harm Millaard, as he is the RAID guru.

Good luck,

Hunt

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Dec 20, 2012 Dec 20, 2012

Copy link to clipboard

Copied

What about using Mac Drive for windows?  I have a Black Magic hyperdeck studio that records to SSD drive (they have to be formatted hsf+ which is a mac format).  I will be docking these in an eSata dock but need to be able to read them with my windows machine.  Does anyone know if Mac Drive slows down the read/write speed?  I could always transfer from the SSD to the internal but was hoping to skip that step and edit directly from the SSD.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Dec 20, 2012 Dec 20, 2012

Copy link to clipboard

Copied

I would post your question to the Adobe Premiere Hardware Forum, as there are several Mac-gurus, there, plus a bunch of folk, who have tested many SSD's, both stand-alone, and in RAID configurations. http://forums.adobe.com/community/premiere/hardware_forum

Good luck,

Hunt

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Jul 12, 2013 Jul 12, 2013

Copy link to clipboard

Copied

BILL

Have a question.   I am going to be loading CS6 MASTER COLLECTION on a new SONY L-SERIES DESKTOP.  The question I have is this - No 2nd internal hard drive for this computer.  Will need to use an external.  Will be doing ESATA or SATA 4TB.  The PC doesn't not come with a ESATA but has ThreeUSB3 connections.  Can I go from the external ESATA connection to one of those USB3's & not die a screaming death while editing in PRIEMERE?  The prcessor is a QUAD CORE 3.80.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Aug 05, 2013 Aug 05, 2013

Copy link to clipboard

Copied

I am not aware of any converter from USB 3.0 to eSATA/SATA, but if you have a 7200 RPM USB 3.0 external, that connects via USB 3.0, you will be almost as fast as an eSATA HDD. Check that you have full USB 3.0 connection, and at least a 7200 RPM disc in the external.

Hunt

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Guest
Nov 23, 2013 Nov 23, 2013

Copy link to clipboard

Copied

Hello Bill Hunt
This is indeed a very good idea, this technology is also currently being accepted by more and more people will become a new trend, I believe a later time technology will become increasingly mature. I do not know when to use an external hard disk to edit video files in my original software can use it? And my DR. iPhone Data Recoveryhttp://www.driphonedatarecovery.com/software can use it?


Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Nov 27, 2013 Nov 27, 2013

Copy link to clipboard

Copied

Andrew,

I am not clear on which Adobe program you are using with your external HDD.

Also, as mentioned above, the speed of the connection to that external HDD will determine how well it will work.

As I often migrate Projects in Premiere Pro, I have EVERYTHING for that Project on the external HDD - including the Media, the Project file and the Scratch Disks. That has worked very well.

As for your backup software, I am unfamiliar with it, so cannot comment.

Hunt

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Feb 26, 2014 Feb 26, 2014

Copy link to clipboard

Copied

Hello Hunt

V.useful post for someone new to HDD issues like myself. Thanks.

Newbie question:

I have 2 internal drives on the "built" system I bought a few years back.

c drive: Samsung SpinPoint F1642 640 GB.

d drive: Storage 2: Samsung SpinPoint T166 500 GB

As I understand it, the aim is to minimise risk of data loss by "spreading" reading/writing of data amongst as many drives as possible.

So I should read from one drive and write to the other?

I have cs5.5 using premiere (music video editing) and soon aftereffects (compositing)

When funds allow an additional HDD, is there any reason why it would have to be physically external to the machine? My Gigabyte GA-P55A-UD3R motherboard has slots for at least 1 internal drive.

If you say yes external is needs to write end video what do you think of the Lacie 301543 D2 Quadra 2000 GB External? My motherboard has an eSATA-300 interface.Logical choice?

Thanks again.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Feb 27, 2014 Feb 27, 2014

Copy link to clipboard

Copied

The primary purpose of using multiple drives is to spread the I/O (Input/Output) over separate drives. This speeds up the writes and reads greatly. Having more drives available, will still increase speed, but to a lesser degree, than going from one drive, up to two.

With two internal physical drives (no partitions), I would have a setup like this:

C:\ OS, programs (including Premiere), Windows Virtual Memory (Page File)

D:\ Projects, Media, Premiere Scratch Disks, Output/Export Files

With three internal physical drives, I would do similar, but with these changes:

C:\ OS, programs (including Premiere), Windows Virtual Memory (Page File)

D:\ Projects, Premiere Scratch Disks, Output/Export Files

E:\ Media Files

Note: one could swap the Media Files on E:\ for the Output/Export Files

With four internal physical drives, I would do similar, but with these changes:

C:\ OS, programs (including Premiere), Windows Virtual Memory (Page File)

D:\ Projects, Premiere Scratch Disks

E:\ Media Files

F:\ Output/Export Files

Now, if a RAID 0, or similar array comes into play, I would use those for E:\ & F:\ in the above scheme.

If one had, say one SSD, I would use it for C:\.

Now, with external drives, so long as the connection is fast, say FW-800, eSATA or USB 3.0, those could take up the slack, if one only had two internal physical drives, so in the last iteration, E:\ and F:\ could be fast externals.

Note: such I/O organization is not so much for safety, but for speed. For safety, I keep my original Video and Still Image Files (plus music, and SFX) on my NAS drive, which is in a RAID array, with reduncancy. I ONLY work with copies of those files, and Copy them from the NAS to the appropriate Project folder.

Hope that this helps make the decision on how to use the computer's drives, whether all are internal, or whether externals figure into the mix. As many of my Projects get migrated between separate computers, I sacrifice some speed, by having every Project on a fast external, along with a Copy of all Media, and all Scratch Disks. This means that I can just carry that external to my workstation, from my laptop, and when I Open that Project, I have everything there. If there is no migration, then the above layouts would provide the greatest performance.

Hunt

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Feb 28, 2014 Feb 28, 2014

Copy link to clipboard

Copied

Thanks Hunt,  That was a comprehensive and immensely useful response.

Peace

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Mar 02, 2014 Mar 02, 2014

Copy link to clipboard

Copied

You are most welcome.

Happy editing,

Hunt

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Jan 15, 2015 Jan 15, 2015

Copy link to clipboard

Copied

Hi Bill,

I'm coming a bit late to the party with the discussion of hard drives for video editing, but I would like to join in. I recently acquired Adobe CC and will be working with my laptop.

I have a PC laptop running Windows 7 64bit, an i7 quad core 2.3GHz CPU, 16gb RAM, Nvidia GT 650m with 2gb VRAM and one internal SSD (500gb Samsung 840 Pro). I have two external drives: a Western Digital 4tb My Book Duo (2-2tb drives defaulted to RAID0) and an external 128gb Samsung 850 Pro. My laptop can only accommodate one internal drive.

The internal SSD tests at 560 MB/s, the My Book Duo at 260 MB/s, and the external SSD at 220 MB/s.

Specifically to your comments at #12 above, how would you recommend I configure my set up?

Thanks in advance.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Jan 18, 2021 Jan 18, 2021

Copy link to clipboard

Copied

Hey Hunt,

Just wanted to briefly express my gratitude for this extremely informative post. It was exactly what I was looking for.

Thank you so much!

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Jan 18, 2021 Jan 18, 2021

Copy link to clipboard

Copied

Alex,

 

Bill Hunt is long gone.  I miss him.  His explanation of multiple drives was very good when he wrote it.  The speed, and now lower cost, of solid state drives (SSD) has changed many of his assumptions.   Having a single, large enough SSD for program, project and source files is probably a better configuration.  External HDDs are always good for storage, backups and archived projects.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Expert ,
Jan 18, 2021 Jan 18, 2021

Copy link to clipboard

Copied

LATEST

I agree with Bill... I build my own (see inside link below) and an M.2 for boot and SSD for files works well

 

https://community.adobe.com/t5/premiere-elements/testing-my-new-build-with-4k-video/m-p/11750996?pag...

 

I do have a USB3 external, but only for backups https://www.amazon.com/StarTech-com-External-docking-station-drives/dp/B00U8KSLA8/

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines