39 Replies Latest reply: Jan 12, 2014 6:19 PM by JSS1138 RSS

    FR:  Project Specific Cache

    JSS1138 Community Member

      I think it would be great if PP didn't require Media Cache files to do what it does, but...for now they're a necessity.  The problem is that occasionally those Cache files go wonky and they need to be deleted so that PP can rebuild them.  And that's where things get tricky.

       

      It's probably not a bad thing to completely delete the contents of both the Media Cache Files and the Media Cache folders every now and then, if for no other reason than to clean up space on the hard drive.  Those files do tend to build up.  But when you have a problem and need to delete only the files for a specific project, things are not quite as easy.  If you work with AVCHD or other DSLR media, you often end up with a multitude of clips labeled identically (00001, 00002, etc.) with no way to know which clip belongs to which project.  And even when you do work with uniquely named media, the files names are often random and locating which clips belong to which project would be a major PITA.

       

      We could set the location for those files to project specific folders manually.  This would make finding the project's Cache files easy, but then every time we changed projects, we'd have to go in and change those settings.  That, too, is an unworkable option.

       

      So here's the proposal.  Premiere Pro would create a project folder for each project's Cache files, like it does with Preview folders.  If you set your Preview folder to a single location for every project, you will notice that PP automatically generates Project folders within that single location.  This makes it very easy to locate and delete a specific project's previews.  I suggest the same for the Cache files, thus making equally as easy to delete only the Cache files for the project we're having trouble with, and not requiring every project to rebuild its cache, only the problem project.

       

       

      ------------------------------------------------------------------------------------------ ------------------------------------------------

       

      Here's the request I ended up with, if anyone cares to copy/paste:

       

      https://www.adobe.com/cfusion/mmform/index.cfm?name=wishform

       

      *******Enhancement / FMR*********

       

      PROJECT SPECIFIC CACHE

       

      I'd like to get the option to store Cache files in project specific folders, much like Preview files.  Possibly via the addition of a checkbox under the current "Save Media Cache files next to originals..."

       

      Why is this feature important to you?

      Cache files often need rebuilding.  Right now there's no way to delete only the problem files.  We have to delete them all, which means even those that aren't a problem will need rebuilding when we reopen those projects.  This is wasted time waiting.

        • 1. Re: FR:  Project Specific Cache
          Steven L. Gotz Community Member

          That makes sense.

           

          My preview files and cache are saved to a drive named "Render and Cache".

           

          On that drive, in the directory named "Adobe Premiere Pro Preview Files", I have three sub directories. One with that same name again, and one named "Media Cache". So why do I also need one named "Media Cache Files"?

           

          Only in the preview files sub directory are there folders for each project. Why not do the same for cache? It seems like it would be easy enough. In fact, it would not hurt my feelings if the preview files and cache resided in the same exact directories.

          • 2. Re: FR:  Project Specific Cache
            Fuzzy Barsik Community Member
            So here's the proposal.  Premiere Pro would create a project folder for each project's Cache files, like it does with Preview folders.

            That would work in a simple scenario. However, with e.g. Dynamic Link involved things are a bit more complicated.

            • 3. Re: FR:  Project Specific Cache
              Steven L. Gotz Community Member

              Fuzzy,

               

              Why? If the Premiere Pro cache is in folders, After Effects and Speedgrade should both be able to handle that, assuming that they even use that cache. They should have individual folders for each project as well.

               

              What am I missing?

              • 4. Re: FR:  Project Specific Cache
                SteveHoeg Adobe Employee

                What I'd really like to see us do is add user controllable policies for cache expiration (eg automatically purge when cache gets to X size, or automatically purge older than X).

                • 5. Re: FR:  Project Specific Cache
                  Steven L. Gotz Community Member

                  You know, I have never paid much attention to the cache. I wipe out the preview files every time I am done with a project. The cache just sits there until I don't have any projects going. Then I reformat that drive.

                   

                  To be honest, other than the .cfa and .pk files, I don't really know what the cache is for! I know how to research it, I am just not sure I care. Should I care? And if so, why?

                  • 6. Re: FR:  Project Specific Cache
                    Fuzzy Barsik Community Member
                    What am I missing?

                    How should they know where to look for cache files? Especially in scenario when one starts from AE? Or in scenario when several PrPro projects derive from a larger one? Duplicate cache files in every single project folder?

                    • 7. Re: FR:  Project Specific Cache
                      Steven L. Gotz Community Member

                      Fuzzy,

                       

                      The files exist. We are just discussing how to organize them. I don't know that Ae needs to see the Pr cache files or vice versa.

                       

                      Take a look at Ae files, They are in folders named in a hexadecimal fashion. Why not by comp?

                      • 8. Re: FR:  Project Specific Cache
                        Fuzzy Barsik Community Member
                        I don't know that Ae needs to see the Pr cache files or vice versa.

                        That's exactly what you're missing. Cache files are currently shared between applications. Different projects, be they AE or/and PrPro projects, can utilise the same media files and, therefore, share the same cache files.

                        • 9. Re: FR:  Project Specific Cache
                          Steven L. Gotz Community Member

                          Yes, but if Ae knows where to find it in the Premiere Pro cache folder, wouldn't it know where to find it if they put it into a sub directory?

                           

                          All of the Ae files are in sub directories.

                           

                          I suppose only the software engineers would know for sure.

                          • 10. Re: FR:  Project Specific Cache
                            Fuzzy Barsik Community Member

                            Steven, how on Earth AE could know in which project subfolder inside Media Cache database to look for, if you start building your dynamically linked compositions from AE, not from PrPro? Search every subfolder and if it finds Media Cache database entry (index) and cache files for a particular media file, then use them, while otherwise create Media Cache entry and build cache files in its own subfolder? Then what's the use of creating those subfolders, if a project actually utilises cache files from different subfolders?

                             

                            If you propose to duplicate Media Cache database entries and cache files for every single project, then that eliminates their sharing.

                            • 11. Re: FR:  Project Specific Cache
                              Steven L. Gotz Community Member

                              Hmmm. Well, you might be right. Unless there was a database keeping track of the files. I imagine that Adobe could store the locations of the cache associated with various media files and all apps would have access to that database.

                               

                              But then the issue of deleting them all comes back to us because the database could be corrupted.

                               

                              Sigh!

                               

                              i am not at all certain that there is a good solution. But maybe Adobe could come up with something.

                              • 12. Re: FR:  Project Specific Cache
                                Fuzzy Barsik Community Member

                                Media Cache Database entries are those .mcdb files, which are stored in the Media Cache folder. They could contain records, which projects utilise this particular media. This way a user could have options to delete all media cache database entries and cache files associated with a working project, delete only those database entries and cache files, which are not exploited by other projects or collect all media cache database entries and cache files associated with the working project so as to e.g. easily transfer it to another machine.

                                • 13. Re: FR:  Project Specific Cache
                                  JSS1138 Community Member
                                  How should they know where to look for cache files?

                                   

                                  How do they know now?

                                  • 14. Re: FR:  Project Specific Cache
                                    Fuzzy Barsik Community Member

                                    Now all cache files are located in a single shared folder.

                                    • 15. Re: FR:  Project Specific Cache
                                      JSS1138 Community Member

                                      user controllable policies for cache expiration

                                       

                                      The only issue I can foresee with that is when the files get deleted and a user has to wait for them to be rebuilt when opening a project.  That's fine if the user purposefully deleted those files, maybe a little frustrating if it happens automatically, even under user set conditions.

                                      • 16. Re: FR:  Project Specific Cache
                                        JSS1138 Community Member

                                        Now all cache files are located in a single shared folder.

                                         

                                        Only if they're set that way.

                                         

                                        Perhaps a single database folder, using project folders for the actual Cache files, which are the ones we often need to delete and let rebuild.

                                        • 17. Re: FR:  Project Specific Cache
                                          Fuzzy Barsik Community Member

                                          And in which way you can set Media Cache Database on CC?

                                          When you change the location for Media Cache Files and/or Media Cache Database in any application, it is changed globally for all applications within the Suite. 'Save Media Cache files next to original when possible' hardly ever worked for me.

                                          • 18. Re: FR:  Project Specific Cache
                                            JSS1138 Community Member

                                            You can change the location manually for each project.  It's not practical, but that will put each project's files into it's own folder.  The point is for Premiere Pro to do that on its own.  As the database connects the media with the Cache files, what I'm suggesting seems like it would be workable using a single database, but project specific Cache folders for the actual files.

                                             

                                            Make sense?

                                            • 19. Re: FR:  Project Specific Cache
                                              Fuzzy Barsik Community Member
                                              Make sense?

                                              Sorry Jim, I don't follow you...

                                              If PrPro would create a subfolder for cache files for every PrPro project, then you have to decide whether to duplicate cache files for the same media files, if it is used by another project, be it PrPro or AE one, or put up with the scattering cache files over different subfolders, if you want to keep sharing them.

                                               

                                              As I mentioned in my comment #12, the only way for keeping cache files shared between applications and having the ability to control what to delete if necessary, as I see it, is to add records in .mcdb files about which project files utilise this particular media. That's the sort of 'Bins vs Metadata' approach.

                                              • 20. Re: FR:  Project Specific Cache
                                                JSS1138 Community Member

                                                I see your point.  I normally use media in only one project, but I can see the difficulty where media is used across multiple projects.

                                                 

                                                Well, there are two possible solutions I can see here.  The first is to duplicate the Cache files.  This keeps things organized on the hard drive making it very easy to delete only those files for a specific project, either because you're done and want to clean house, or because there's an issue and you need to rebuild them.  The advantage here is very easy file management outside of PP.  The disadvantage is potential increased usage on the hard drive.  (Given the affordability of storage these days, this is not a bad option.)

                                                 

                                                The second option is as you suggested in post 12, sort of a turbo-charged version of the Clean button that we have now.  The advantage is potentially less usage on the hard drive.  The disadvantage is the project needs to be open in order to clean house, you can't do it manually with a file manager, which is normally my preference for this sort of thing.  (I'm of the mind that any NLE should be able to read a file, or create a new file, but should have zero ability to rename, move or delete any files on the hard drive.  That kind of thing should always be done outside the NLE with a proper file manager.)

                                                 

                                                So we have two options to make life easier when Cache files need rebuilding.  I'd love to hear from others which they prefer.

                                                 

                                                1. Project Specific Cache folders.  (Easy file management, but potential for duplicate Cache files on the hard drive.)

                                                 

                                                2. TurboClean (No duplicates, but same current mess of file management and project has to be open to use it.)

                                                 

                                                I'm partial to option 1, but honestly I think either would make things better.

                                                • 21. Re: FR:  Project Specific Cache
                                                  Fuzzy Barsik Community Member
                                                  The disadvantage is the project needs to be open in order to clean house, you can't do it manually with a file manager.

                                                  Well, yes, with file manager a user won't be able to filter out Media Cache database entries (yes, they also need to be deleted or collected) and cache files, related to a particular project. However, opening a project for cleaning Media Cache database (collecting files) is not required. That's rather a matter of developing interface. An example is Dynamic Link dialog, where a user can select a project. Such interface can be however advanced, allowing to select a single project or several ones. Etc.

                                                  • 22. Re: FR:  Project Specific Cache
                                                    JSS1138 Community Member

                                                    It's not a bad idea, in fact it's a good one.  But something tells me we have a better chance of getting this done using either option 1 or 2 as stated, rather than Adobe developing a new app for the task.

                                                     

                                                    Just my gut feeling on this one.

                                                    • 23. Re: FR:  Project Specific Cache
                                                      Fuzzy Barsik Community Member

                                                      No, no, no, it shouldn't be a separate application or utility, this interface should be incorporated into Preferences -> Media tab of all applications, which deal with Media Cache database. Particularly, PrPro, AE and AME. Moreover, it's a good practice to clean up all auxiliary data by means of an application, which creates those auxiliary data, be they files, folders in various locations or system registry entries. A mere user shouldn't bother where those data are located so as to manually delete them via lower level utilities such as file manager or registry cleaner. Any lower level operations are normally for advanced users, who can solve issues that are not resolved via standard means. That's my probably not too humble opinion. And I think Adobe could start from cleaning junk .xmp files, which remain after deleting preview files and have to be deleted manually.

                                                      • 24. Re: FR:  Project Specific Cache
                                                        JSS1138 Community Member

                                                        I'm confused.  You suggest in post 21 that opening PP to delete cache won't be necessary, then you say in 23 that the feature will be built into the Preferences.  That requires opening PP, which is the disadvantage of option 2.

                                                        • 25. Re: FR:  Project Specific Cache
                                                          Fuzzy Barsik Community Member

                                                          Nope, I said opening a project is not necessary, which means database entries and cache files for a particular project can be deleted without opening it. And no, I don't think that cleaning Media Cache database and cache files via PrPro, AE or AME is a disadvantage. On the contrary, it's an advantage in my point of view.

                                                          • 26. Re: FR:  Project Specific Cache
                                                            JSS1138 Community Member

                                                            I said opening a project is not necessary

                                                             

                                                            You can't get to the Preferences without opening a project, so if the ability was in Preferences, you'd need to open a project.  And to work well, I think you'd need to open the project whose Cache files you want to delete, not just any random or even empty project.

                                                            • 27. Re: FR:  Project Specific Cache
                                                              Fuzzy Barsik Community Member

                                                              You actually can get to the Preferences without opening a project in both AE and AME, and I thought that annoying compulsory PrPro welcome screen was removed on CC as well. Regard to opening a project so as to check whether it is empty or not, well, you can if you want. But as I pointed out, it is not necessary - all data in external XML files, which are both .prproj and .mcdb, can be read without opening them.

                                                              • 28. Re: FR:  Project Specific Cache
                                                                JSS1138 Community Member

                                                                AME doesn't have 'projects', and AE opens in an empty project.  So...you're still opening a project.

                                                                 

                                                                But how do you envision PP would know which Cache files you want to delete without opening that project?  Remember, the goal is to delete only the Cache for a specific project, not all of them.

                                                                • 29. Re: FR:  Project Specific Cache
                                                                  Fuzzy Barsik Community Member

                                                                  AE doesn't open an empty project. It opens an empty workspace, in which you can start building a project, if you want. Or you can simply tweak application settings without creating a project. But I wouldn't go into semantics.

                                                                   

                                                                  Regard to how, I already proposed to add records into .mcdb files. References to projects, which utilise this particular media, will allow to filter out respective Media Cache database entries and cache files.

                                                                  • 30. Re: FR:  Project Specific Cache
                                                                    JSS1138 Community Member

                                                                    I mean on the user end.  How do we tell Premiere Pro to delete only the Cache files from the Smith wedding and not any from the Jones wedding?  What kind of user interface do you see without opening the Smith wedding so PP knows which files you want to delete?

                                                                    • 31. Re: FR:  Project Specific Cache
                                                                      Fuzzy Barsik Community Member

                                                                      Since I'm away from my workstation, I can't portray it with screenshots and will try in words. As I mentioned earlier, Dynamic Link dialog allows user to choose any project. And when one picks it up, the list of compositions (sequences) inside the project can be observed. Similar dialog can be exploited in this case.

                                                                       

                                                                      When a project is chosen, the application scans Media Cache database, filters out respective files, warns the user if this or that media is shared between projects and offers several options such as delete all files or just those, which are not utilised by other projects, etc.

                                                                      • 32. Re: FR:  Project Specific Cache
                                                                        JSS1138 Community Member

                                                                        Honestly, that sounds like a lot more work to implement than just making separate folders.

                                                                        • 33. Re: FR:  Project Specific Cache
                                                                          Fuzzy Barsik Community Member

                                                                          First and foremost that offers a 'solution' for shared media. And second, we have different opinions on how auxiliary data, created for an application internal use only, should be handled.

                                                                          • 34. Re: FR:  Project Specific Cache
                                                                            JSS1138 Community Member

                                                                            Given the affordability of storage, the likelihood that the majority media used by PP editors is probably project specific, and the additional work required to implement on Adobe's part, I'm not sure this 'solution' is all that necessary.

                                                                            • 35. Re: FR:  Project Specific Cache
                                                                              Fuzzy Barsik Community Member
                                                                              Given the affordability of storage, the likelihood that the majority media used by PP editors is probably project specific

                                                                              Well, that's just an assumption. In our pipeline the amount of projects, which do not share media, equals zero. Moreover, I guess sharing cache files is not just about decreasing disk space, but also about speed of data exchange between applications.

                                                                              • 36. Re: FR:  Project Specific Cache
                                                                                JSS1138 Community Member

                                                                                So perhaps a new checkbox below the one for "Save Media Cache files next to originals" labeled "Use project folders for Media Cache files."  Likely pretty simple to code, and solves the file management issues that sparked the request, but doesn't force it for those users that share a lot of media between projects.

                                                                                • 37. Re: FR:  Project Specific Cache
                                                                                  Fuzzy Barsik Community Member

                                                                                  Just for the record: 'Save Media Cache files next to originals' currently implies creating Media Cache database entry(ies) in Media Cache folder, some auxiliary files in Media Cache Files folder and .cfa and .pek files in the same folder, where the media file is located. If this media file is used in AE project, .cfa and .pek are also created in the same folder, where AE project file is located. So, currently that is the worst option 'file management'-wise.

                                                                                  • 38. Re: FR:  Project Specific Cache
                                                                                    JSS1138 Community Member

                                                                                    That's why I don't use it.  Cache files should go on their own separate drive.  But...it would help greatly if those files were easy to manage, preferably using Explorer or Finder.  Lumping them all into one folder makes it nigh impossible to do so.

                                                                                    • 39. Re: FR:  Project Specific Cache
                                                                                      JSS1138 Community Member

                                                                                      Here's the request I ended up with, if anyone cares to copy/paste:

                                                                                       

                                                                                      https://www.adobe.com/cfusion/mmform/index.cfm?name=wishform

                                                                                       

                                                                                       

                                                                                       

                                                                                      *******Enhancement / FMR*********

                                                                                       

                                                                                      PROJECT SPECIFIC CACHE

                                                                                       

                                                                                      I'd like to get the option to store Cache files in project specific folders.  Possibly via the addition of a checkbox under the current "Save Media Cache files next to originals..."

                                                                                       

                                                                                      Why is this feature important to you?

                                                                                       

                                                                                      Cache files often need rebuilding.  Right now there's no way to delete only the problem files.  We have to delete them all, which means even those that aren't a problem will need rebuilding when we reopen those projects.  This is wasted time waiting.