-
1. Re: Any News On The LR Leakage Problem
Lee Jay-ZyZk56 Feb 22, 2011 7:10 AM (in response to grandpahenry)I am not seeing memory leaks on XP SP3 or Win 7.
-
2. Re: Any News On The LR Leakage Problem
clvrmnky Feb 22, 2011 7:13 AM (in response to grandpahenry)The question is, did you report and receive confirmation that there was a significant memory leak under the circumstances you identified?
Otherwise, the answer is "probably not."
-
3. Re: Any News On The LR Leakage Problem
Jasonized Feb 22, 2011 7:51 AM (in response to grandpahenry)So far, I'm still having memory problems with Mac, and also keep the status tool open so I can watch. When it starts swapping, I reboot.
3.3, so far, is slower at eating all the memory, but it still does. And I doubled my ram to 16 gig because 3.x was so much more hungry than 2.7...
I'm hoping it will be fixed, but haven't heard anyone say they are actually looking for it since 3.2. So, they may have decided that a half a day of use is good enough. Sigh.
Cheers!
-
4. Re: Any News On The LR Leakage Problem
Stephen_Carpenter Mar 2, 2011 3:50 AM (in response to grandpahenry)grandpahenry wrote:
2. I am considering upgrading to a more robust computer using a multicore Intel I7, 6GB memory, and an SSD work disk running under Windows 7, 64 bit. Is anyone having the leakage problem in that envoronment?My system has Intel Core i7-720QM CPU, 8GB mem, 60GB SSD + 500GB HD & Win7 64 bit.
Two days ago I was doing some light editing of some old photos and the LR3.3 memory usage was about 500MB which is fine. I iconised LR while still in develop mode and this morning I noticed that it was consuming 3GB. That's 2.5GB of memory growth in about 12 hours of uptime while it was iconised. I'd say there are still memory leak issues.
-
5. Re: Any News On The LR Leakage Problem
clvrmnky Mar 2, 2011 6:00 AM (in response to Jasonized)I run Lr continuosly on my Mac, nearly from the moment I login. I never logout or restart (uptime is measured in weeks, usually) and my total in-use memory hovers around 4-5GB. I jump back and forth between Develop and the Library Grid, and make heavy use of keywroding, smart collections and publish services.
What would help is identifying what gestures appear to spike "real" mem, and confirmation that Lr releases that real mem when other apps need it. What are the sizes of the files in /var/vm and so on. What Lr modules are active (they are generally not loaded until you invoke them) and if any custom plugins are installed.
Identifying memory issues is more of a black art than science. There are many kinds of memory, so the idea is to find out what is being used, how it is being used over time, and how the virtual memory system on the OS is handling the requests from the kernel and userland processes.
-
6. Re: Any News On The LR Leakage Problem
clvrmnky Mar 2, 2011 6:11 AM (in response to Stephen_Carpenter)Stephen_Carpenter wrote:
grandpahenry wrote:
2. I am considering upgrading to a more robust computer using a multicore Intel I7, 6GB memory, and an SSD work disk running under Windows 7, 64 bit. Is anyone having the leakage problem in that envoronment?My system has Intel Core i7-720QM CPU, 8GB mem, 60GB SSD + 500GB HD & Win7 64 bit.
Two days ago I was doing some light editing of some old photos and the LR3.3 memory usage was about 500MB which is fine. I iconised LR while still in develop mode and this morning I noticed that it was consuming 3GB. That's 2.5GB of memory growth in about 12 hours of uptime while it was iconised. I'd say there are still memory leak issues.
It is typical for Lr to consume a fair amount of memory, even when idle. You don't say what kind of memory you are talking about (i.e., real vs. shared, or the Windows equivalents) and the process monitor on Windows is less than helpful in reporting these numbers in a meaningful way.
That being said, the idea of a leak is that an app grows unbounded. Unless there is sufficient interest in physical memory by other apps, the OS will satisfy any request Lr makes, even if it means growing the footprint over time. 8GB is still a lot of memory when you consider a typical request is often only a few bytes long. The Lua garbage collector is not deterministic, so (again) unless some heuristic has been met dead objects may not have been collected, or some concrete representations (files handles &c.) may not have been finalized.
So, the first thing to determine is if:
- Real memory use is truly growing unbounded, with physical memory not being released by the app such that the OS needs to swap to satisfy other requests.
- There is significant indication that this memory is actually affecting the app, or other apps it is sharing this memory with.
The problem is that it is not easy to verify to any real degree the answers to these questions without the proper tools and techiques. It is nearly impossible to tell by doing a cntrl-alt-del and looking at the activity monitor (or whatever it is called on WIndows.)
That is, the fact that an app has requested and received some large amount of real or shared memory is not really significant. Memory is intended to be used. The question is whether or not this memory is also being shared, and if Lr is somehow not being a good neighbour.
Again, the best thing to do is contact Adobe Support with this. These user-to-user forums are not the best place to hash this stuff out, mostly because determining memory issues requires dedication you won't probably find from a volunteer.
-
7. Re: Any News On The LR Leakage Problem
Jasonized Mar 2, 2011 6:27 AM (in response to clvrmnky)That is, the fact that an app has requested and received some large amount of real or shared memory is not really significant. Memory is intended to be used. The question is whether or not this memory is also being shared, and if Lr is somehow not being a good neighbour.
I've had a problem with memory allocation since 3.0. The beta did not seem to absorb memory the way the actual release did, but to be honest, I didn't check as I ran it only occasionally. So it might have had the problem as well. 3.x on my system will consume all 16 gigs of ram on my Mac Pro, and then start swapping. If I exit LR, memory will not be released, but other programs can request and get access to it. LR itself does not seem to re-use memory, and hence starts swapping. This behaviour is repeatable and reproducable. 3.2 had a larger problem, and I could reproduce it more quickly, but 3.3 still has the problem. It just takes about twice as long before I run out.
Really a pain having to reboot my machine all the time. I have to keep the memory icon up, and when it gets close to full, have to reboot to "fix" it.
I do not think this is appropriate behaivour for a program, and 2.7 did not exhibit this behaiviour. So yes, IMO, LR still has a memory problem somewhere!I just wish it would be fixed soon enough. Sadly though, Adobe people have stopped saying they are still looking for memory problems with the last dot release. So, all I can do is hope..
Cheers!
-
8. Re: Any News On The LR Leakage Problem
grandpahenry Mar 2, 2011 6:31 AM (in response to clvrmnky)When I originally posted this I had 2GB running LR 3.3 using XP Pro SP3 32 bit. On 3 occasions LR, which was the only app running, used memory to the point where it choked the system and I had to do a hardware reset to get the system up and running. I have since increased the system memory to 4 GB (3.5 usable to XP) and have been able to run LR 3.3 for up to an hour without significant memory growth.
In my case I have a dual monitor system, use one monitor for the LR window and use the 2nd monitor for close up looks at noise and other artifacts. Going 1:1 seems to cause a jump in memory use. LR should not use so much memory as to cause a lockup. I thought I left those behind with Windows 95/98.
I am currently working with a couple of integrators to build me a system around a Supermicro board, Intel X7 Extreme with 6 cores, 12 threads, a OCZ 480 SSD SATA3, and 6GB memory. I am going to use a Matrox 9128 graphics card to support my current 22" monitors at 1680 X 1050 until I replace them with NEC 27" PA271W because of the wide gamut of color and a resolution of 2560 x 11440.
Would be interested in any comments on the configuration.
-
9. Re: Any News On The LR Leakage Problem
clvrmnky Mar 2, 2011 6:40 AM (in response to Jasonized)If I exit LR, memory will not be released, but other programs can request and get access to it. LR itself does not seem to re-use memory, and hence starts swapping.
The first part is normal on a Mac. The OS assumes correctly that an app may be restarted and will want the same libs and so on. So memory remains in use until an allocation suggests otherwise.
The second part is explained by an app that does many small allocations. Not a lot of memory is being allocated and held onto. This is typical in many multithreaded apps that work on a lot of little things concurrently.
-
10. Re: Any News On The LR Leakage Problem
Stephen_Carpenter Mar 2, 2011 7:24 AM (in response to grandpahenry)grandpahenry wrote:
I am currently working with a couple of integrators to build me a system around a Supermicro board, Intel X7 Extreme with 6 cores, 12 threads, a OCZ 480 SSD SATA3, and 6GB memory. I am going to use a Matrox 9128 graphics card to support my current 22" monitors at 1680 X 1050 until I replace them with NEC 27" PA271W because of the wide gamut of color and a resolution of 2560 x 11440.
Would be interested in any comments on the configuration.
Considering the spec of the system I think you should go for 12GB memory (3x4GB).
-
11. Re: Any News On The LR Leakage Problem
Jasonized Mar 2, 2011 8:07 AM (in response to clvrmnky)The second part is explained by an app that does many small allocations. Not a lot of memory is being allocated and held onto. This is typical in many multithreaded apps that work on a lot of little things concurrently.
I'm sorry, but I do not believe that with 16gig of memory one program should invoke swap and slow everything down. I agree that lots of memory is being allocated. Infact, just previewing 3 images (grid on one monitor, and full size on second monitor) will (once it's swapping) cause another swap to occur. If a program such as LR has 16 gigabytes in memory allocated, it should be giving up the oldest memory rather than swap. That's just silly...
And 2.7 did not exhibit this 'feature', and worked quite handily. And I was not forced to reboot all the time, either. I left LR open for weeks at a time. Now I can not even rate/edit 800 images without rebooting (or suffering a massive slow down). So, I disagree that this behavior is "normal" or acceptable. 2.7 worked just fine with only 8 gigs of ram; 3.3 suffers with 16.
Cheers!
-
12. Re: Any News On The LR Leakage Problem
clvrmnky Mar 2, 2011 11:02 AM (in response to Jasonized)Jasonized wrote:
The second part is explained by an app that does many small allocations. Not a lot of memory is being allocated and held onto. This is typical in many multithreaded apps that work on a lot of little things concurrently.
I'm sorry, but I do not believe that with 16gig of memory one program should invoke swap and slow everything down. I agree that lots of memory is being allocated. Infact, just previewing 3 images (grid on one monitor, and full size on second monitor) will (once it's swapping) cause another swap to occur. If a program such as LR has 16 gigabytes in memory allocated, it should be giving up the oldest memory rather than swap. That's just silly...
Upon re-reading my comments, you'll find I never made any claims beyond suggesting that not everything you think is a problem is actually a problem. I described the usual source of some specific symptoms that are unique to the OS X virtual memory system.
For example, my comment above (which you have edited such that it is no longer in context) suggests that the behaviour you describe could actually be described by a very typical pattern. I did not volunteer if this pattern is to be expected or not, or is wanted or not. It very well may be a pathological case.
But as we do not have enough information to determine this, who knows? But I know for a fact that there is a know pattern with many small allocations that can look like this. Remember that Lr is not a 100% native app. There is a non-deterministic garbage collector and finalizer that is in control of everything, and understanding how the VM profiles over time is the only way to get a clear picture of what is going on.
Please be careful not to over-simplify modern virtual memory systems. There very well be a problem here, but some of what you describe is business as usual. If you have actually profiled the application, and have logged swap statistics that show something interesting why not share it with Adobe? Otherwise, all you are doing is guessing, which helps exaclty no one, since few people seem to be able to reproduce what you are seeing.
Your assumption about old allocations needing to be released is actually not true. There is a highly compicated scheduler that manages virtual memory
Plainly put, you may have convinced yourself where the problem lies. I am way too wary of real-world memory problems to be sure you have convinced me. We have no details, no confirmation and no steps to reproduce. It is not your responsibility to provide this, but even so, that fact that it is missing means there is probably little a user forum can do for you.
I will recap (since I expect few will take the time to read through all of this): there very well may be a problem with the application regarding native allocations, or releasing nil objects or finalizing unused resources. If you want to help determine if this is a problem you must contact Adobe Support. It is the ONLY way real progress can be made on corner-cases like this.
-
13. Re: Any News On The LR Leakage Problem
Jasonized Mar 2, 2011 11:21 AM (in response to clvrmnky)I do not disagree with you about simple things such as garbage collection. Yes, I believe modern memory management can handle allocating millions of small objects with a reasonable memory usage (well, we could when I worked at Cadence, where millions of allocations and deallocations were the norm). And 2.7 did not exhibit this issue. So something in the 3.x releases has changed enough that memory allocation and deallocation has been severly impacted, at least for some of us. Since 3.0, 3.1 and 3.2 all had memory issues tweaked or fixed, I don't see having yet another as impossible.
I don't believe I edited too much of your comment, since I took the whole paragraph as a separate subject, but I apologize if you feel I did. It was not my intention to wrongly attribute anything to you.
I do not know where the problem lies; it has something to do with previewing images, from the method to reproduce it. Nor did I say anything about not reusing nil objects; I said it should release memory or reuse it before being forced into swap. In fact, if you go back and view previews that were previewed earlier, it does not increase memory, it seems to reuse the memory already allocated to that object. My complaint is that using more than 16 gigabytes to store previews (or whatever it's doing) for viewing is ridiculous; when you are forced into swap, that slows things down even more than releasing a memory image (writing the old one, reading the new one is far more expensive than release and read a new one). I would think a reasonable amount of scratch ram could be used; unlimited (well, at least more than 16gig) is too much.
I have put in carefully written bug reports for each iteration of 3.x, to duplicate the problem. I have not once been contacted by Adobe regarding further attempts to duplicate or track down the bug. I would be happy to help, if asked. Since I haven't been, I resopnd to people asking about memory problems in the hope that if nothing else, it will keep the subject in someones forebrain instead of the trash heap.
This one issue made me seriously consider using 2.7 for most of my work, as at least I didn't have to reboot all the time. After 3.3 came out, and doubling my memory, it's livable if still highly annoying.
Cheers!
-
14. Re: Any News On The LR Leakage Problem
Ian Lyons Mar 2, 2011 12:49 PM (in response to Jasonized)Jasonized wrote:
I do not know where the problem lies; it has something to do with previewing images, from the method to reproduce it.
I don't think it's a simple as "something to do with previewing images" otherwise this forum would be full of similar reports. I have an 8-core Mac Pro with 16GB ram and it's been running now for just over 16 days without restart. There hasn't been a day in that period that I haven't been building previews, yet I've not experienced memory leaks, swaps, etc. At the moment it's building previews and free memory is again hovering between 100MB and 400MB with Lr using just under 1.5GB real memory and another 3.6GB as cached memory. Attached screenshots form Lr System Info screen show this and screenshot from Mac OS X Activity Monitor show the current split of memory across active, inactive, wired etc. Note that even after 16 days I've only used 15MB for swap and Page Outs are sitting at 32.9MB.
Given that you're seeing a problem that others with similar specs are not would suggest that there's something going on in your system that's causing Lr/OS to misbehave. Unfortunately, I have no clue as to what that might be, but removing things like plug-ins would be one of my first sanity checks.
-
15. Re: Any News On The LR Leakage Problem
Jasonized Mar 2, 2011 2:16 PM (in response to Ian Lyons)It probably isn't something "simple". I understand that, even if I did imply it. :}
But I also see from your images that all of your memory is in use, and have swapped a bit. Interesting, in that you say LR has only used 5 gig of ram. Who is using the rest of the 11 gig? The inactive ram is attached to LR, in that if you go view a previously viewed image, the memory will move back from inactive to active. This indicates that LR has a cached image (of some sort) in that memory.
So, can you really say LR only uses 5 gig? Maybe only 5 gig active, but it is keeping the inactive memory hostage as well, since it will swap instead of reuse inactive memory. So, if you keep previewing new images, does it keep swapping? This is the problem I've reported, mine will continue to swap. Do you feel that swapping continually is a valid responce to examining images in a system with 16 gig of ram, when 11 gig is inactive? Since you seem to have a machine similar to mine (drives and displays probably vary), what happens to your swap if you examine, say, another 200 images? How much time are you wasting while your system swaps? Not to mention, of coruse, what happens when you bring up PS...
I notice it most when I import images from a day's shoot, say, 800 images, and don't want to sit and wait for it to continually swap after viewing and rating half the images...
It's interesting that you don't attribute swapping because LR has all of your memory hostage as a problem... I haven't used my machine hard enough to cause swapping before 3.x came out!
Cheers!
-
16. Re: Any News On The LR Leakage Problem
Jasonized Mar 2, 2011 2:30 PM (in response to Ian Lyons)Oh yes, and until very recently, I didn't have any plugins at all in LR... just your basic stock system. I did add two plugins recently, but did not seem to affect behaviour.
Cheers! -
17. Re: Any News On The LR Leakage Problem
Ian Lyons Mar 2, 2011 3:27 PM (in response to Jasonized)Jasonized wrote:
So, can you really say LR only uses 5 gig? Maybe only 5 gig active, but it is keeping the inactive memory hostage as well, since it will swap instead of reuse inactive memory. So, if you keep previewing new images, does it keep swapping? This is the problem I've reported, mine will continue to swap. Do you feel that swapping continually is a valid responce to examining images in a system with 16 gig of ram, when 11 gig is inactive? Since you seem to have a machine similar to mine (drives and displays probably vary), what happens to your swap if you examine, say, another 200 images? How much time are you wasting while your system swaps? Not to mention, of coruse, what happens when you bring up PS...
The inactive memory is managed by the OS, not Lr. Sure, Lr probably has images, whatever stored in the inactive memory, but it's not hogging it for itself. You asked about Ps. Well, I just launched Photoshop CS5 upsized an image to 1.25GB, did few bits and bobs to it and the inactive dropped back to just over 2GB. At the same time, Lr was building 1:1 previews for 1800 5DMk2 files. Safari and Outlook were also running. Photoshop was taking 6.5GB, Lr still working around 5GB and the rest was being used by the other apps, Wired and Free. It was only when I doubled the Ps image to 3.5GB that the swap value increased, as did Page Outs, but what the heck should I expect with Ps pulling close to 9GB by this stage. Yeh, and inactive was all but gone. I then closed Ps and left things for a few minutes. Attached screenshot is where the system is at now. This tells me that Lr isn't holding onto anything that it shouldn't and the OS is doing pretty much what it's supposed to do. Given what I've just done I think the Page Out increasing to 675MB isn't surprising.
FWIW, it used to irritate me that Inactive always seem to be high after Lr had built previews, but then again Bridge does exactly the same thing when building previews. However, the Bridge, Ps and Lr engineers tell it's normal for some processes to act in this way. It's only a problem when the Inactive memory can't be brought back into play, and since I'm not seeing that I stopped being irritated.
-
18. Re: Any News On The LR Leakage Problem
Jasonized Mar 2, 2011 8:49 PM (in response to Ian Lyons)I suppose you could say my real complaint boils down to LR causing swap all by itself. I see no real reason it should be doing that, and 2.7 didn't. So saying it's all the OS's problem doesn't make complete sense. Something changed between 2.7 and 3.x, in the way LR manages memory requests. 2.7 handly ran for weeks without exiting on 8 gigs of ram, and no swapping issues. 3.x doesn't, at least on my machine. Maybe you don't notice it because you run more apps at the same time than I do? The mac is my editing box, not my general purpose box. So I tend to run only LR by itself for weeks before I start doing other things like PS and printing. Maybe that's why I notice it so much more.
And, when PS closed, all the memory came back. But when you close LR, none of it does. Well, okay, it could be because PS is using it as active, but it seems some other mechanism is also at work here. I would be happy enough thaf if you closed LR the memory was released, as then I could just restart LR instead of restarting my machine in order to stay out of swap.So perhaps a work around would be to bring up PS and use up a large chunk of ram brieflly, so LR will reuse instead of swap. I'll give that a try next time, and see if that "fixes" the problem. Faster than rebooting.
Thanks Ian!
Cheers!
-
19. Re: Any News On The LR Leakage Problem
grandpahenry Mar 3, 2011 7:36 AM (in response to Jasonized)Is there anyone from Adobe listening on this? If so, why are you folks not addressing this issue?
-
20. Re: Any News On The LR Leakage Problem
Jasonized Mar 3, 2011 9:38 AM (in response to grandpahenry)Is there anyone from Adobe listening on this? If so, why are you folks not addressing this issue?
Hmm....
Well, judging from what Ian wrote: Adobe engineers apparently don't seem to think this is a problem.
It's only a problem when the Inactive memory can't be brought back into play, and since I'm not seeing that I stopped being irritated.
Since LR is the only (user) program running, and it (memory) isn't being brought back into play, I don't seem to be able to stop being irritated. :}
Any program that claims to use only 5-6 gigabytes that causes swapping on a 16 gigabyte system has a flaw, in my mind. Apparently the way I use the machine makes the issue far larger than someone who only uses one computer for everything. Doesn't mean it's not a problem though.
Marking memory inactive and available for reuse by other programs is not the same as making it free; I would far rather "suffer" through having to recreate the preview than swap to disk. It would be faster...
How about something like this? Put a sane limit on the amount of memory LR is marking inactive, something like half the system memory? That would stop it from going into swap gratuitously, and would make it stop bothering people like me...
Cheers!
-
21. Re: Any News On The LR Leakage Problem
MikeLeone Mar 3, 2011 5:28 PM (in response to grandpahenry)Is there anyone from Adobe listening on this? If so, why are you folks not addressing this issue?
This is a user forum. Some Adobe folks stop in, from time-to-time, but
this is not an official support, or request, forum. No doubt they have
an official internal list of problems to address. And perhaps they are
addressing it - doing internal testing and re-coding. When they have an
answer/fix, they will (hopefully) include it in a point release, as a
bug fix, or in the next official release.
But until the fix is ready and tested, issuing status reports on the
progress of the work is not likely to happen.






