Cluster Help Needed

[Neo Rye]

I have a cluster that contains 2 instances on a Windows 2008 server. It seems that when one of the instances max out it's memory, that the other stops taking requests. In the end, they both crash.  Is this expected behavior? I thought the were suppose to be isolated from each other? Thoughts?

-Thanks!

[speedpedal]

Need more info:

1. Are you using Jrun Clustering, or other types?

2. What is your memory allocation for the instances?

3. what is your total RAM size?

4. CF version?

5. what is your memory allocation for each instance?

Can you post the entry on your server.logs which states the "OutOfMemory" issue?

We can start here.

[Neo Rye]

Need more info:

1. Are you using Jrun Clustering, or other types?

A: Yes, I'm using JRun and building the instances and cluster through the CF Admin interface.

2. What is your memory allocation for the instances?

A: I currently have 2 instances with 3.5GB allocated per instance.

3. what is your total RAM size?

A: The server has 16GB of RAM running Windows 2008 and IIS7.5 on a dual quad core box.

4. CF version?

A: CF9

Can you post the entry on your server.logs which states the "OutOfMemory" issue?

A: I'll see if I can find the OutOfMemory log, but I'm running FusionReactor to monitor the instances and can watch it run out. I originally had 3 instances with 2048MB of memory each, but that was getting maxed out, so I reduced it to 2 instances and increased their memory to the 3.5GB. I'm now in the 70-75% memory threshold and that's fine and everything is holding. When I had the 3 instances and 1 of those instances maxed out it's memory in FR, than the other 2 instances would stop handling requests too. When this happens, the box still has plenty of memory to spare, so I'm not starving the server.

I was wondering if their was some lower level relationship in a cluster, like the IIS connector that would be affected by one of the instances running out of memory or threads? I can stop 1 of the instances and the other instance will continue to run and handle requests fine. It just seems like a memory issue or perhaps the memory issue creates a thread backup and that's causing an issue on the other instance. I'd think that one instance could crash and not crash the other and that's my goal, even if the crash was a memory or thread issue.

I can re-create this if I turn on server monitoring and enable memory tracking for just one of the instances. Eventually the memory gets all used up and both instances stop responding.

-Thanks!

[Guest]

try turning of file based session replication for a while and see if it has an impact

else:

• check ur code if it is calling external resources when it fails - they may be down
• check ur code to see if it is serializing large objects
• as above, but in sessions?

btw, with 2 in a cluster, if one fails, naturally the other takes all the load and will probably fail, 3 is a better number