17 Replies Latest reply on Dec 12, 2010 8:53 AM by ablichter

    Camera Raw cache size and location

    Yammer Level 4

      I've been using ACR for about 3 years now, and today it occurred to me that I'm not that sure I know what the cache is actually used for. I mean, I know what a cache is, but what is ACR caching, and why?

       

      I use Bridge for most of my photo work, displaying high quality thumbnails and previews, often full-screen ("monitor-sized previews" by default). Bridge has its own cache, which I understand contains jpegs of thumbnails, previews, and 100% views. Does ACR's cache have any bearing on Bridge's behaviour and performance, especially with thousands of raw images?

       

      What sort of size and location of ACR cache should I be using - what factors do I need to consider?

        • 1. Re: Camera Raw cache size and location
          ablichter Level 2

          When a RAW first time was opened by ACR, the first part of the processing (decoding, decompressing, linearizing, and demosaicing) is stored in the ACR cache.

          Your specific ACR settings are either stored in the Camera Raw Database => C:\Users\username\AppData\Roaming\Adobe\CameraRaw\Database or in xmp sidecar files.

          Does ACR's cache have any bearing on Bridge's behaviour and performance, especially with thousands of raw images?

          Depends.

          When Bridges renders its own cached previews, it needs to call ACR to do so, by this ACR data is written to the ACR cache.

           

          So as long there are already cached previews in the Bridges cache, it seems the ACR cache is not touched anymore and might be deleted. But have in mind that only 240px and 1024px previews are cached by default in Bridges cache. The full screen previews are generated on the fly when chosing full screen and you zoom in. So latest than the ACR cache is rebuild.

           

          The default ACR cache size, which is 1GB, should be able to keep the cached data for 200-250 RAW of a better camera (5D, 50D, D300/700 ~4,25MB each here). Seems the data created by LR 3.3 beta for each RAW is almost double the size.

           

          Even when I use Bridges not much, I have pointed Bridges and the ACR cache to another fast harddisk. I just hate it when software creates caches and temp-files in c:\users\username\blabla ;-)

          1 person found this helpful
          • 2. Re: Camera Raw cache size and location
            Yammer Level 4

            ablichter wrote:

             

            When a RAW first time was opened by ACR, the first part of the processing (decoding, decompressing, linearizing, and demosaicing) is stored in the ACR cache.

            Your specific ACR settings are either stored in the Camera Raw Database => C:\Users\username\AppData\Roaming\Adobe\CameraRaw\Database or in xmp sidecar files.

            Does ACR's cache have any bearing on Bridge's behaviour and performance, especially with thousands of raw images?

            Depends.

            When Bridges renders its own cached previews, it needs to call ACR to do so, by this ACR data is written to the ACR cache.

             

            So as long there are already cached previews in the Bridges cache, it seems the ACR cache is not touched anymore and might be deleted. But have in mind that only 240px and 1024px previews are cached by default in Bridges cache. The full screen previews are generated on the fly when chosing full screen and you zoom in. So latest than the ACR cache is rebuild.

             

            The default ACR cache size, which is 1GB, should be able to keep the cached data for 200-250 RAW of a better camera (5D, 50D, D300/700 ~4,25MB each here). Seems the data created by LR 3.3 beta for each RAW is almost double the size.

             

            Even when I use Bridges not much, I have pointed Bridges and the ACR cache to another fast harddisk. I just hate it when software creates caches and temp-files in c:\users\username\blabla ;-)

            Thanks for your comments.

             

            That's interesting about the cache contents. I had wondered what was actually stored, as raw images appear to be processed on the fly (hence the warning triangle on opening). It makes sense to do demosaicing once only, as this should not change (unless you're going from 5.6 to a newer version).

             

            I don't understand how you come to the figure of 4.25MB per image though. Wouldn't a 12Mp 14-bit RGB raster image require about 60MB? That would allow for only 16 images in a 1GB cache.

             

            I currently have both ACR and Bridge caches on a separate drive to the photos. However, even though my (approx) 20,000 photos have supposedly been cached and indexed, Bridge continues to occasionally generate thumbnails and previews without good reason, taking 2-3 seconds per image with my current defaults. I always assumed this behaviour was purely down to Bridge - somehow deciding that it's existing cache needed updating.

             

            I have Bridge set to generate HQ thumbnails on demand (although I usually manually generate a cache after upload anyway), monitor-sized previews, keep 100% previews, have cache size set to maximum, and the NEF files' settings are stored as XMP sidecars. I have instructed Bridge to cache every folder on my photo drive too, but it still insists on generating thumbnails/previews every now and then, without any pattern.

             

            So, I guess there's no need to have the ACR cache any bigger than necessary to work on a single batch of images from a shoot, or a collection of images. Indeed, I suppose it's possible that a cache can be too big, and slow down the retrieval process.

            • 3. Re: Camera Raw cache size and location
              ablichter Level 2

              I don't understand how you come to the figure of 4.25MB per image though. Wouldn't a 12Mp 14-bit RGB raster image require about 60MB? That would allow for only 16 images in a 1GB cache.

              Because that was what I saw yesterday ;-) A 40D RAW is about 10-14MB. Idon't know how and what ACR is storing in this files but they are ~4.39MB only for my 40D. Of course what's in cache won't be a full developed image, morelikely the instructions how to interprete a pixel or something similar.

              Yesterday I was on a different computer with D70 NEFs.

              2010-12-04_130854.jpg

              The *. dat and the index.dat is what makes the ACR cache.

              The 14.9MB file is from a Canon 7D, which RAW have 8Mpx more than those of my 40D. So the size of the cached data might vary a lot ;-)

              I pointed ACR and Bridges cache to the same partition, in case you wonder. Which is no problem.

              1 person found this helpful
              • 4. Re: Camera Raw cache size and location
                Yammer Level 4

                Ah, I see what you mean now. In my case, my D300 (12Mp) is producing 4.5MB files. I never thought that each file represented a single raw image, as the file names don't relate to the images, but I suppose it's obvious when you think about it.

                • 5. Re: Camera Raw cache size and location
                  Curt Y Level 6

                  ablichter wrote:

                   

                  The default ACR cache size, which is 1GB

                  Hi, butting in on this thread.  I was under the impression the ACR cache was used only for the data base option rather than the XMP sidecar option.  In the later option the data would be stored in central cache.   Is that right or wrong thinking?

                   

                  Also, the exported cache is useful when exporting files, but does it help in speeding up thumbnail generation in day to day operations, or does it slow it down as disk has to write in 2 locations?

                   

                  Can you clearify?

                   

                   

                   

                  Thanks

                  • 6. Re: Camera Raw cache size and location
                    ablichter Level 2

                    Hi,

                      I was under the impression the ACR cache was used only for the data
                    base option rather than the XMP sidecar option.  In the later option the
                    data would be stored in central cache.   Is that right or wrong
                    thinking?Is that right or wrong thinking?

                    When I understand you right, than your thinking is wrong. ;-)

                     

                    ACR cache and ACR database / XMP options are seperate functions. As said the cache is meant for avoiding decompressing and demosaicing RAW again and again, f.e. once when Bridge parses a folder with RAW to build its own cache and previews, and a second time, when you "open" them by ACR either via PS LR or Bridge.

                     

                    ACR development settings are written either to the database or to XMP.  You can delete ACRs cache and your develope settings still are left intact and applied in Bridge or when opening the RAW in ACR. Of course at the same time a cache entry would be build for that RAW, because there is none anymore.

                     

                    At least that is how I put my observations together.

                    Also, the exported cache is useful when exporting files, but does it
                    help in speeding up thumbnail generation in day to day operations, or
                    does it slow it down as disk has to write in 2 locations?

                    Hope I get you right here. What is meant with exporting files?

                     

                    If you mean Bridges "automatically export cache to folders if possible"  feature? This is meant for situation when some shares the images on a netwok.

                     

                    I don't now anything about the structure of this folder related caches, but you will have two files in any of your image folders. But all files which are lie about at disks might degrade performance sooner or later. Depends of course how often a disk will be defragmentated, but I prefer something like one database on a own partition with large clusters, which is used for cache only so the fragmentation stays small as possible.

                    • 7. Re: Camera Raw cache size and location
                      Yammer Level 4

                      So, I've worked out that (on my system) a 10GB ACR cache will serve 2300 images.

                       

                      Assuming I have limitless disk space, is there any point my increasing the cache size to accomodate all my 20k photos? Or is there a downside to a very large ACR cache?

                       

                      I already have my Bridge cache set to maximum, as I hate waiting for thumbnail & preview generation (increasingly slower these days, with ACR becoming more complex). The maximum is labelled "500,000" items, although it doesn't say what an "item" is - I assume it's a cached jpeg of a  thumbnail, preview and/or 100%. Properties of the cache directory say that there are 46,000 files in 1600 folders, so I'm still not sure why it still generates thumbnails & previews every now and then (there are allegedly about 22,000 image files on my photo drive).

                      • 8. Re: Camera Raw cache size and location
                        ablichter Level 2

                        btw: I did parsed the same folders/images on my laptop, and the dat-files get 4.25MB large. Might be that special settings are getting into account, monitorprofile or so, but this is just a guess.

                        Assuming I have limitless disk space, is there any point my increasing the cache size to accomodate all my 20k photos? Or is there a downside to a very large ACR cache?

                        Yeah, you have 20k more files on disk. ;-)  If this has an impact depends if they are are on a dedicated disk or not...

                        Personally I don't think I need all my RAW cached, at least not the older ones. I'm pretty fine with space for the last ~ 3.000 ones. If for a reason the cache is full, the oldest will be overwritten by the new ones anyway ;-)

                        so I'm still not sure why it still generates thumbnails & previews every now and then

                        If you have 22.000 RAW, you should have 66.000 previews. 240px,1024px and fulls screen. But as said I believe full screens are generated on demand only, means when using the slideswhow or the loupe tool. You can see that in the statusbar, if there is no full size preview yet, you'll get "one full size extraction", when you use the loupe tool.

                        although it doesn't say what an "item" is

                        Guess its everything for which a preview can be generated or can hold metadata, like PDFs, Indesign files..

                        • 9. Re: Camera Raw cache size and location
                          Yammer Level 4

                          I have Bridge set to keep 100% previews, but they are not generated automatically, so I should have 2 "items" per image in the cache plus a few more, so the figure suggest everything has been cached.

                           

                          I've set my ACR cache to a much lower value than before. I decided there's not much point keeping files more than a few weeks old. Bridge, on the other hand, is the crucial one for speeding up my workflow, and I think ACR cache settings doen't really help. If anything, keeping the ACR defaults simple will be the best thing I can do, to speed up the generation of unnecessary cache images (i.e. no v6 clever stuff).

                           

                          So, lots of discussion, but no firm recommendations.

                          • 10. Re: Camera Raw cache size and location
                            ablichter Level 2
                            So, lots of discussion, but no firm recommendations.

                            Sorry. Thought you will be fine with the facts and as I can see, you are able to draw the right conclucions for you / your workflow. F.e. reagarding the ACR cache.

                            How exactly I can be of help, after all you know yet, because this

                            What sort of size and location of ACR cache should I be using - what factors do I need to consider?

                            was answered? Just let me know.

                             

                            And please take into account I'm my mothertongue is not english, so I might don't get anything.

                            So I have my problem to understand this

                            to speed up the generation of unnecessary cache images (i.e. no v6 clever stuff).

                            To speed up things I would recommend to make an own partition on the fastest disk around, ~ 10GB. Format it with 64kb cluster size and point Bridges cache to it. Before doing the later move the actuall content to it.And use it only for cache, maybe as well for the ACR cache.

                            I believe there can't be more done to speed Bridge up.

                            Regarding Bridges cache we are not talking about RAW files only, right?

                             

                            Bridge is not the most stable app around. Means there are circumstances on which you need to reset preferences and/or purge the cache. F.e. when you change the monitor profile, because you use to profile your monitor every six months.

                            Maybe another app will fit your requirements more than Bridge can do? Depends what you are using Bridge for? I don't know yet.

                            • 11. Re: Camera Raw cache size and location
                              Yammer Level 4

                              ablichter wrote:

                               

                              How exactly I can be of help, after all you know yet, because this
                              What sort of size and location of ACR cache should I be using - what factors do I need to consider?

                              was answered?

                               

                              I'm still not sure what size of cache I should use, and what are the advantages and disadvantages of bigger or smaller ACR caches.

                               

                              So I have my problem to understand this
                              to speed up the generation of unnecessary cache images (i.e. no v6 clever stuff).

                               

                              Bridge uses ACR to generate its raw image cache. This process is fairly processor-dependent, especially now that ACR uses more sophisticated processing in version 6, and, depending on ACR defaults, this can be significantly slower than in previous versions. Bridge appears to generate previews unnecessarily sometimes, because every image is already cached.

                               

                               

                              To speed up things I would recommend to make an own partition on the fastest disk around, ~ 10GB. Format it with 64kb cluster size and point Bridges cache to it. Before doing the later move the actuall content to it.And use it only for cache, maybe as well for the ACR cache.

                              I believe there can't be more done to speed Bridge up.

                              Regarding Bridges cache we are not talking about RAW files only, right?

                               

                              Bridge is not the most stable app around. Means there are circumstances on which you need to reset preferences and/or purge the cache. F.e. when you change the monitor profile, because you use to profile your monitor every six months.

                              Maybe another app will fit your requirements more than Bridge can do? Depends what you are using Bridge for? I don't know yet.

                              I already use a dedicated drive for application caches, because of the fragmentation issue. Your suggestion of a larger cluster size is very interesting, and I may try this when I have some spare time.

                               

                              I only use Bridge for managing photos: predominantly raw, but also some PSD and JPEG.

                               

                              Why would I need to reset the cache with a new monitor profile? Surely this is not applied to cache JPEGs?

                               

                              Bridge has just received an update. Since then, I am yet to spend a full day with it, but I did download Lightroom 3 a couple of weeks ago, after Bridge started to send me insane with it's preview generation, background disk activity and "cannot write metadata" messages. Supposedly Lightroom's catalogue system is superior, but I am yet to be convinced that putting all my XMP data in one basket is a good idea.

                              • 12. Re: Camera Raw cache size and location
                                MadManChan2000 Adobe Employee

                                The ACR cache is used to accelerate subsequent loads/reads of an image. So if you're flipping through a lot of images and revisiting them, revisits will take less time because the image has been cached. Certain operations like the decompression of a compressed raw image, etc. won't have to be redone.

                                 

                                The bigger the cache, the more of these entries you can hold. So the more images you can flip through without having to repeat the initial internal processing.

                                • 13. Re: Camera Raw cache size and location
                                  Yammer Level 4

                                  Thanks for joining in, Eric.

                                   

                                  So do these 4.5MB files in the ACR cache on my system each represent a 12MP 14-bit raw image? That would mean that a 1GB cache will serve about 230 images.

                                   

                                  Would it make sense to make the cache just big enough to process one or two shoots? Is there any point making it any bigger, and is there such a thing as an over-large ACR cache? (ie. it would slow the system down)

                                  • 14. Re: Camera Raw cache size and location
                                    Curt Y Level 6

                                    Don't know if this relates directly to the issue, but with CS3 era I had more than one user that found relief by upping the 1 gig default to 10 gigs.  They said it increased the speed of what they were doing.

                                    • 15. Re: Camera Raw cache size and location
                                      ablichter Level 2

                                      Hi,

                                      I'm still not sure what size of cache I should use, and what are the advantages and disadvantages of bigger or smaller ACR caches.

                                      Maybe Eric can tell something indeep about the impact of a large cache.  I'm not sure about that myself, but don't believe a larger cache is a real disadvantage, as long the disk on which it reside is maintained: defrag often). Since Bridge and ACR work hand in hand here, it might depend to your workflow and to your hardware. Sorry, I can't give a clear answer here.

                                       

                                      As said I would put it just large enough for a certain number of RAW or some weeks work. Means, when there is a chance I'ld touch a certain RAW in six weeks again, than six weeks * x images per day I shoot.

                                      Not sure if you are a Pro. I'm not and usually have not much new images every month.  I always was fine with the default settings.

                                       

                                      I don't work with Bridge and open one RAW after another in ACR, or maybe 20+ in a row when doing panorama ;-)

                                       

                                      But Bridge has to do a lot of work on the fly when it generates the previews for all new RAW in a folder, like: "read the image from HDD", "process it by ACR, "write Bridge previews and metadata to HDD", "write to ACR cache " - not necessarily in this order.

                                       

                                      And it seems it does this and some additional steps always (not only when first time reading RAW), because it f.e. needs to check if there are already processing information in ACR database or XMP file for a RAW or if they have been created in "meantime". Means ACR settings could have be changed after Bridge did a first time generation of the cache data, maybe because the RAW was openene directly via PS or by another app.

                                      this process is fairly processor-dependent, especially now that ACR uses more sophisticated processing in version 6, and, depending on ACR defaults, this can be significantly slower than in previous versions.

                                      And Bridges eats huge amount of memory as well - seems it holds a lot of information in memory before flushing it to disk.

                                       

                                      After reading this post of yours I made a test with my 12k RAW the other day. I tried to let Bridge create previews for all of them in one step, but it ended up complaining that there is not enough memory after ~ half an hour.

                                      Bridge appears to generate previews unnecessarily sometimes, because every image is already cached.

                                      See above. Bridge/ACR checks for updated processing information in XMP or ACR database and metadata updates. If it find some it might need to rebuild its cached image.

                                      Why would I need to reset the cache with a new monitor profile? Surely this is not applied to cache JPEGs?

                                      You are right, sorry I mixed things up here :-(  I meant ACR camera profiles. Aside that I changed monitor profile a lot over the last weeks doing some tests, and after that Bridge often refused to start until the cache was purged…

                                      Your suggestion of a larger cluster size is very interesting, and I may try this when I have some spare time.

                                      All my disks meant for storing large files (images, music, scratch, temp-file) are formated with a larger cluster size. It speeds things up a bit.

                                      Supposedly Lightroom's catalogue system is superior, but I am yet to be convinced that putting all my XMP data in one basket is a good idea.

                                      ?

                                      Of course LR shares ACR's cache. I don't know much about LR, but I believe XMP are not only kept in one basket here, right? They are written to the images and in case of RAW to XMP files. Otherwise there wouldn't be an interoperability between LR and Bridge. Similar to Bridge LR stores previews in it own "cache" which is as well a folder and file based "database". Check "preferences".

                                       

                                      But as said I use another DAM, in which’s database all data needed for searching is kept, but I also write all metadata (IPTC,XMP) to the images, even for RAW files. By this I don't need a large ACR cache and also don't have XMP files around, which I hate. When saying database, I mean a real database, not a collection of files which are kept on disk. ;-)

                                      Aside other advantages, I can take my database with me on vacations and can work on my images like I do at home. I have all my keywords and categories with me in one file and when back, I just copy the database to my desktop.

                                      • 16. Re: Camera Raw cache size and location
                                        Yammer Level 4

                                        ablichter wrote:

                                         

                                        > Supposedly Lightroom's catalogue system is superior, but I am yet to be convinced that putting all my XMP data in one basket is a good idea.

                                        ?

                                        Of course LR shares ACR's cache. I don't know much about LR, but I believe XMP are not only kept in one basket here, right? They are written to the images and in case of RAW to XMP files. Otherwise there wouldn't be an interoperability between LR and Bridge. Similar to Bridge LR stores previews in it own "cache" which is as well a folder and file based "database". Check "preferences".

                                         

                                        But as said I use another DAM, in which’s database all data needed for searching is kept, but I also write all metadata (IPTC,XMP) to the images, even for RAW files. By this I don't need a large ACR cache and also don't have XMP files around, which I hate. When saying database, I mean a real database, not a collection of files which are kept on disk. ;-)

                                        Aside other advantages, I can take my database with me on vacations and can work on my images like I do at home. I have all my keywords and categories with me in one file and when back, I just copy the database to my desktop.

                                        Lightroom uses its own "catalog" database to store image metadata. You can also set Lightroom to additionally write to XMP files for compatibility with ACR.

                                         

                                        One of the things I like about ACR is that every image gets its own sidecar. Yes, I actually like that. It means that they are always kept together but without writing to the image file, are portable, and are not subject to central database corruption. The downside is that it is probably slower, due to their distributed nature.

                                        • 17. Re: Camera Raw cache size and location
                                          ablichter Level 2

                                          Yammer P wrote:

                                           

                                          Lightroom uses its own "catalog" database to store image metadata. You can also set Lightroom to additionally write to XMP files for

                                          compatibility with ACR.

                                          Okay I see what you mean.

                                           

                                          One of the things I like about ACR is that every image gets its own sidecar. Yes, I actually like that. It means that they are always kept together but without writing to the image file, are portable, and are not subject to central database corruption. The downside is that it is probably slower, due to their distributed nature.

                                           

                                          Likes are different and we need to have backups anyway.

                                          I'm pretty happy I can switch theirs creation off.