13 Replies Latest reply: May 22, 2012 9:23 PM by Jim Curtis RSS

    another one step forward, one step back post

    Jim Curtis Community Member

      Just noticed this on CS6 (Mac):

       

      After I work on a project, I use ChronoSync to backup my project folder (which contains all my media and other assets besides caches) to another drive, which is an interim backup until LTO.

       

      Now, ChronoSync is re-writing ALL my media to the back up disk.  This is a big deal when I have 120GB of media in a project, which isn't unusual.

       

      What was taking a few seconds or minutes is now taking hours.

       

      I can see that Pr is changing the modification date on my media when I open (or maybe save) a project.  This is what ChronoSync is looking at to determine whether to backup that file or not.

       

      I'm not changing the media (not intentionally anyway).  Why the change in the way Pr works?  I don't see an advantage.  Can anybody (especially from Adobe) provide some insight?

       

      I'm guessing that Pr is adding some metadata to my source files, yes?  Is this necessary?  Can I turn it off? 

       

      If Pr is modifying my sources, doesn't this risk corrupting my media files?

        • 1. Re: another one step forward, one step back post
          Jim Curtis Community Member

          This isn't a problem for anybody but me?

           

          Doesn't anybody else do incremental backups?

          • 2. Re: another one step forward, one step back post
            needles27 Community Member

            Hi Jim,

            I haven't used ChronoSync before, instead relying on Time Machine to back up all of my files except the media files.  I put those files into a separate folder which I exlude from the TM backup (And I back up the original media files once to a drobo on ingest so I can always go back to the original camera footage if necessary)  Anyway, you didn't ask about other workflows, but I'm wonderng if an interim solution is to leave all of your media in a separate folder structure that you can ignore with ChronoSync?  Probably not a groundbreaking idea, and it doesn't address the actual issue of why Pr re-writes the modification date....but maybe something to do right now to make sure you have backups.

             

            Now I am curious - can you give me  brief explanation why ChronoSync is better for backup than Time Machine? Thanks.

            • 3. Re: another one step forward, one step back post
              JSS1138 Community Member

              Doesn't anybody else do incremental backups?

               

              Only manually, and even then just the project file itself using Save Copy... to a separate hard drive.

              • 4. Re: another one step forward, one step back post
                Jim Curtis Community Member

                Hi Needles,

                 

                I haven't used Time Machine, but from what I've read, it keeps a lot of files so that you can go back in time, hence the name.  Since I hail from the first days of personal computers and one undo, I've learned to use incremental file names, etc.  So, TM seems to create unnecessary file bloat, given my method.

                 

                The reason I went with ChronoSync is that it has a verify function, which I've found to be useful when copying from media cards with complex file structures.  I've had copies come to me in a corrupt state, and doing the verify pass before I give the card back to the shooter is good peace of mind.

                 

                Another reason is that it does incremental "mirror" backups.  It examines the source and target folders, and will only update what was changed between backups. 

                 

                And this is the issue with Pr CS6.  When I open a project in CS6, all the modification dates get changed to the time I opened the project.  It appears to be happening during the initial scan and load of media.

                 

                I had already thought of your suggestion to segregate my media folder.  And that's a good idea, thanks.  But, that's going to create more work, and I'd rather use a computer to work less, not more.  I like being able to keep all my project assets in one folder, inside all of any given client's projects in one big folder.  And as I mentioned, it all goes to LTO weekly or thereabouts, and I really don't want to be moving folders back and forth, hither and yon, not only for the additional work, but for the potential of screwing something up.

                 

                My system was working just great before, and as I said, if there's not a damn good reason why Adobe changed this behavior (and I sure can't think of one), I'd like to cast my vote for changing it back.

                 

                Do you use Time Machine to back up your media?  If so, doesn't this new wrinkle bloat your drive with constant re-writes of your media?

                • 5. Re: another one step forward, one step back post
                  needles27 Community Member

                  Thanks for the longer description of your workflow. 

                  A couple of notes:

                   

                  As far as the ingesting of footage coming off of cards or HDs from the shoot, I use an app designed especially for this called Shotput Pro.  It is designed to ingest footage, append filenames, copy to multiple locations and verify the data.  If you haven't already, check it out - works great on location for a smooth data process.

                  Time Machine will work in the same way of only doing incremental backups of what has changed since the last scan.  I guess in this way, you would come up against the same issue of the changed modification dates causing havoc.

                  I use Time Machine to back up everything except for the shoot reels, which would quickly fill my TM drive.  After ingesting all of the footage into my file structure, I will copy the Shoot Reels to a Drobo drive where it will be protected as a backup outside of Time Machine.  Since the original media isn't being modified over the course of an edit, I can always retrieve it and relink the files if my HD that I edit from blows up.  (Not to delve too deep into a backup discussion, but if it is really sensitive footage, i will make another copy and store it in my safe deposit box at the bank)  So, that leaves all of the other files that do get backed up to TM hourly, which are generally much much smaller. 

                   

                  I also like to leave everything related to one project in 1 folder.  Your note about not wanting to have to think about exluding folders from backup is something I share - however, I do have to manually tell TM not to backup the shoot reel folder for each project I create.  I haven't found an automated way of doing this yet.

                   

                  I can be more specific about how I organize my project folders and how it fits into a backup workflow, but I don't think that is what you are looking for since you already have your workflow down (with the exception of this wrench Adobe threw into the works...) Only if you are interested i will share my system.

                   

                  To your original question - I agree that the way CS6 modifies all of your footage mod dates is annoying and I wish i had an answer for you.

                  • 6. Re: another one step forward, one step back post
                    Jim Curtis Community Member

                    Just tried something.  I switched off the three bottom checkboxes in the Preferences>Media pane, opened an old project, and it didn't modify the source files (yet).

                     

                    I'm not sure which one did the trick, but it was probably the "Automatically Refresh Growing Files" option, which is new.  Not sure what that's for anyway.

                     

                    Screen shot 2012-05-10 at 2.38.41 PM.png

                    • 7. Re: another one step forward, one step back post
                      Todd_Kopriva Adobe Employee

                      It was the Write XMP ID To Files On Import option that was relevant here.

                       

                      That preference tells Premiere Pro to add a single piece of metadata (a number, basically) to each piece of media so that it can be tracked throughout the pipeline. That helps, for example, when creating media cache files, since Premiere Pro checks that number to see if that piece of media has already been processed.

                      • 8. Re: another one step forward, one step back post
                        Jim Curtis Community Member

                        Thanks, Todd.  What does the "Refresh Growing Files Every ____ seconds" do?

                         

                        Can't wait to get the Help PDF.

                         

                        BTW, another "one step forward" story...  I just got in some 1080p24 footage from Arri Alexa (I think) that's ProRes4444.  Pr creates an ARRI Cinema Sequence when I create from footage.  Scrolling the timeline is still awful.  Only perhaps slightly better than in CS5.5.

                         

                        I've been using Pr CS6 on jobs since I downloaded it on Monday.  SD projects are pretty snappy, as far as timeline shuttling.

                         

                        And before anybody asks about my RAID, it's wicked fast.  No bottleneck there. QT Player zips footage back and forth with alacrity.  I still don't understand why Pr can't shuttle footage thas has no scaling or effects applied with equal ease.

                        • 9. Re: another one step forward, one step back post
                          needles27 Community Member

                          Jim,

                          Glad you got this sorted out.  I'm curious - what RAID system do you have?  I have just striped 2 drives in my MacPro 4,1 and it has been ok, but looking to get a dedicated RAID.  I got really confused about RAID controllers and other hardware needed.  Don't mean to hijack the thread, but since you mentioned it, any advice you can offer?

                          • 10. Re: another one step forward, one step back post
                            Jim Curtis Community Member

                            Needles, I have an DIY 8 - 2TB SAS disk RAID5 in an Enhance E8 box, with an ATTO R680 PCIe card.  I get over 2Gbs reads and writes.  I spent days researching.  In retrospect, I advise most wary people to buy a bundle from a VAR.  They'll get you up to speed (They're complicated.), and stand behind the bundle.  But, if you're good at troubleshooting, you can save money buying the parts separately.  The biggest mistake I made was buying a Highpoint Tech HBA before I got smart and bought the ATTO.  Tech support is very important.  You can PM me for more info.

                             

                            Now, back to our regularly scheduled thread....

                            • 11. Re: another one step forward, one step back post
                              Wil Renczes Adobe Employee

                              Jim Curtis wrote:

                               

                              Thanks, Todd.  What does the "Refresh Growing Files Every ____ seconds" do?

                               

                              That's a broadcast feature - for systems like Omneon MediaDecks, where you can be generating an MXF on the fly (say, an incoming satellite feed), we allow you to edit the file while it's still actively being captured.  That pref is the refresh interval at which PPro will update its snapshot of the MXF details, ie the available media duration.  This setting only actively applies to MXF files that it detects as incomplete (ie no partition footer).  It should have no bearing on completed MXF files, or on any other media type, for that matter.

                               

                              Cheers

                              • 12. Re: another one step forward, one step back post
                                needles27 Community Member

                                Jim, have you had any issues since you unchecked Write XMP ID To Files On Import ? Your post has inspired me to dive into Chronosync and I am wondering if:

                                a) Your always-updating file date situation has resolved

                                b) Any metadata or database issues cropped up since you don't have that option checked?

                                • 13. Re: another one step forward, one step back post
                                  Jim Curtis Community Member

                                  Hey Needles,

                                   

                                  Same as it ever was now.  So, a) Yes the always-updating files has ceased since unchecking Write XMP ID... and b) no new issues that I can report on.  I've developed a style of editing that works for me that doesn't involve logging or using any metadata.  So, if there were issues with the data, I might not even know about them.

                                   

                                  I hope you enjoy using ChronoSync.  It has features I don't use, such as scheduling.  I only use it for mirroring with verification.  And the log comes in handy occasionally. 

                                   

                                  I have backups spread over several disks, so what I do is save Syncronizer files for each client or folder, and open it after each session (using the Recent Files menu), and click Synchronize.  It's not as hands-off as Time Machine, but it's pretty darn low maintenance after I get my Syncronize files set up.