1 Reply Latest reply: Aug 28, 2014 12:20 AM by WASIL RSS

    LiveCycle unable to access Cache Controller. Exception message - Error accessing the cache container - Error on GET action for cache Replicated:UM_CLUSTER_INVALIDATION_CACHE

    Asadullah Community Member

      The incident starts after all the members are removed from the DCS Stack DefaultCoreGroup view and added back again by WebSphere Application Server. Following exceptions are seen after LiveCycle was added back in again to the view.

      We have a clustered environment with two nodes 2A and 2B. Server 2A crashed and therefore all members on 2B node were removed from the DCS view. Later the new core group view was installed but LiveCycle did resume operations as expected.

       

      Errors below:

      Exception caught while dealing with cache : Action - Get, ObjectType - UM_CLUSTER_INVALIDATION_CACHE, Exception message - Error accessing the cache container - Error on GET action for cache Replicated:UM_CLUSTER_INVALIDATION_CACHE

       

      ServiceRegist W   Cache get failed for service EncryptionService Reason: Error accessing the cache container - Error on GET action for cache

       

      Error accessing the cache container - Error on GET action for cache Local:SERVICE_FACTORY_CACHE

       

      The following message appeared for several different services:

      Cache put failed for service [CredentialService, ReaderExtensionsService,EncryptionService, etc]

       

       

      SSOSessionCle W   Error in cleaning stale sessions from the database. These sessions would be deleted in next trigger

                                       java.lang.RuntimeException: com.adobe.livecycle.cache.CacheActionException: Error accessing the cache container - Error on ENTRYSET action for cache Local:UM_ASSERTION_CACHE

        at com.adobe.idp.um.businesslogic.authentication.AssertionCacheManager.removeExpiredResults( AssertionCacheManager.java:125)

        at com.adobe.idp.um.scheduler.SSOSessionCleanupJob.execute(SSOSessionCleanupJob.java:69)

        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)

        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529)

      Caused by: com.adobe.livecycle.cache.CacheActionException: Error accessing the cache container - Error on ENTRYSET action for cache Local:UM_ASSERTION_CACHE

        at com.adobe.livecycle.cache.adapter.GemfireLocalCacheAdapter.entrySet(GemfireLocalCacheAdap ter.java:219)

        at com.adobe.idp.um.businesslogic.authentication.AssertionCacheManager.removeExpiredResults( AssertionCacheManager.java:99)

        ... 3 more

      Caused by: com.gemstone.gemfire.distributed.DistributedSystemDisconnectedException: GemFire on 2B<v1>:9057/51073 started at Sun Aug 17 08:57:23 EDT 2014: Message distribution has terminated, caused by com.gemstone.gemfire.ForcedDisconnectException: This member has been forced out of the distributed system.  Reason='this member is no longer in the view but is initiating connections'

        at com.gemstone.gemfire.distributed.internal.DistributionManager$Stopper.generateCancelledEx ception(DistributionManager.java:746)

        at com.gemstone.gemfire.distributed.internal.InternalDistributedSystem$Stopper.generateCance lledException(InternalDistributedSystem.java:846)

        at com.gemstone.gemfire.internal.cache.GemFireCacheImpl$Stopper.generateCancelledException(G emFireCacheImpl.java:1090)

        at com.gemstone.gemfire.CancelCriterion.checkCancelInProgress(CancelCriterion.java:59)

        at com.gemstone.gemfire.internal.cache.LocalRegion.checkRegionDestroyed(LocalRegion.java:669 4)

        at com.gemstone.gemfire.internal.cache.LocalRegion.checkReadiness(LocalRegion.java:2587)

        at com.gemstone.gemfire.internal.cache.LocalRegion.entries(LocalRegion.java:1815)

        at com.gemstone.gemfire.internal.cache.LocalRegion.entrySet(LocalRegion.java:7941)

        at com.adobe.livecycle.cache.adapter.GemfireLocalCacheAdapter.entrySet(GemfireLocalCacheAdap ter.java:209)

        ... 4 more

      Caused by: com.gemstone.gemfire.ForcedDisconnectException: This member has been forced out of the distributed system.  Reason='this member is no longer in the view but is initiating connections'

        at com.gemstone.org.jgroups.protocols.pbcast.ParticipantGmsImpl.handleLeaveResponse(Particip antGmsImpl.java:106)

        at com.gemstone.org.jgroups.protocols.pbcast.GMS.up(GMS.java:1289)

        at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:767)

        at com.gemstone.org.jgroups.protocols.VIEW_SYNC.up(VIEW_SYNC.java:202)

        at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:767)

        at com.gemstone.org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:276)

        at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:767)

        at com.gemstone.org.jgroups.protocols.UNICAST.up(UNICAST.java:294)

        at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:767)

        at com.gemstone.org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:625)

        at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:767)

        at com.gemstone.org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:187)

        at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:767)

        at com.gemstone.org.jgroups.protocols.FD_SOCK.up(FD_SOCK.java:504)

        at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:767)

        at com.gemstone.org.jgroups.protocols.FD.up(FD.java:438)

        at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:767)

        at com.gemstone.org.jgroups.protocols.Discovery.up(Discovery.java:258)

        at com.gemstone.org.jgroups.stack.Protocol.passUp(Protocol.java:767)

        at com.gemstone.org.jgroups.protocols.TP.handleIncomingMessage(TP.java:1110)

        at com.gemstone.org.jgroups.protocols.TP.handleIncomingPacket(TP.java:1016)

        at com.gemstone.org.jgroups.protocols.TP.receive(TP.java:923)

        at com.gemstone.org.jgroups.protocols.UDP$UcastReceiver.run(UDP.java:1320)

        at java.lang.Thread.run(Thread.java:773)

       

       

      [22/08/14 0:28:10:237 EDT] 00000b4f WorkPolicyEva W   Exception thrown in background thread. Cause: java.lang.NullPointerException

      [22/08/14 0:29:10:252 EDT] 00000b4f WorkPolicyEva W   Exception thrown in background thread. Cause: java.lang.NullPointerException

      [22/08/14 0:30:10:268 EDT] 00000b4f WorkPolicyEva W   Exception thrown in background thread. Cause: java.lang.NullPointerException

      [22/08/14 0:31:10:283 EDT] 00000b4f WorkPolicyEva W   Exception thrown in background thread. Cause: java.lang.NullPointerException

      [22/08/14 0:32:10:298 EDT] 00000b4f WorkPolicyEva W   Exception thrown in background thread. Cause: java.lang.NullPointerException

      [22/08/14 0:33:10:313 EDT] 00000b4f WorkPolicyEva W   Exception thrown in background thread. Cause: java.lang.NullPointerException

      [22/08/14 0:34:10:328 EDT] 00000b4f WorkPolicyEva W   Exception thrown in background thread. Cause: java.lang.NullPointerException

      [22/08/14 0:35:10:343 EDT] 00000b4f WorkPolicyEva W   Exception thrown in background thread. Cause: java.lang.NullPointerException

      [22/08/14 0:36:10:358 EDT] 00000b4f WorkPolicyEva W   Exception thrown in background thread. Cause: java.lang.NullPointerException

      [22/08/14 0:37:10:373 EDT] 00000b4f WorkPolicyEva W   Exception thrown in background thread. Cause: java.lang.NullPointerException

      [22/08/14 0:38:10:389 EDT] 00000b4f WorkPolicyEva W   Exception thrown in background thread. Cause: java.lang.NullPointerExceptionor

       

      I have tried looking for the root cause about why LiveCycle was not able to resume normally, didn't find anything related.