13 Replies Latest reply on Oct 26, 2018 7:58 AM by chung yonge16845482

    Can you help me work cache activation?

    chung yonge16845482 Level 1

      Hello Everyone,

       

      I have implemented executeCurlCommand using curl command to delete / activate cache.

       

      Now cache deletion works well.

       

      But cache activation doesn't work.

       

      I have no idea why.

       

      If you are availabe, can you help me work cache activation?

       

      Here are my source code to delete / activate cache using curl command.

      -------------------------------------------------------------------

       

      public static final String DISPATCHER_HEADER_CQ_ACTION = "CQ-Action";

      public static final String DISPATCHER_HEADER_CQ_HANDLE = "CQ-Handle";
      public static final String DISPATCHER_HEADER_CQ_PATH = "CQ-Path";
      public static final String DISPATCHER_HEADER_CONTENT_LENGTH = "Content-Length";
      public static final String DISPATCHER_HEADER_CONTENT_TYPE = "Content-Type";
      public static final String DISPATCHER_HEADER_CONTENT_TYPE_VALUE = "application/octet-stream";

       

      public static void execute(BundleContext bundleContext, Event event) {

              if (bundleContext != null && event != null) {

                  final ServiceReference rrServiceReference = bundleContext.getServiceReference(ResourceResolverFactory.class.getName());

                  final ResourceResolverFactory resolverFactory = (ResourceResolverFactory) bundleContext.getService(rrServiceReference);

                  final ResourceResolver resolver = AdminService.getResourceResolver(resolverFactory);

                  final ReplicationAction replicationAction = ReplicationEvent.fromEvent(event).getReplicationAction();

                  final ReplicationActionType replicationActionType = replicationAction.getType();

                  if (replicationActionType.equals(ReplicationActionType.ACTIVATE)) {

                      executeCurlCommand(getDispatcherUris(resolver), replicationAction.getPaths(), ReplicationActionType.DELETE);

                      executeCurlCommand(getDispatcherUris(resolver), replicationAction.getPaths(), ReplicationActionType.ACTIVATE);

                  } else if (replicationActionType.equals(ReplicationActionType.DEACTIVATE) ||

                          replicationActionType.equals(ReplicationActionType.DELETE)) {

                      executeCurlCommand(getDispatcherUris(resolver), replicationAction.getPaths(), ReplicationActionType.DELETE);

                  }

              }

      }

       

      private static void executeCurlCommand(List<String> dispatchers, String[] paths, ReplicationActionType methodType) {

              for (String path : paths) {

                  for (String dispatcher : dispatchers) {

                      String[] command = {

                              "curl",

                              "-H", Constants.DISPATCHER_HEADER_CQ_ACTION + ": " + methodType.getName(),

                              "-H", Constants.DISPATCHER_HEADER_CQ_HANDLE + ": " + path,

                              "-H", Constants.DISPATCHER_HEADER_CQ_PATH   + ": " + path,

                              "-H", Constants.DISPATCHER_HEADER_CONTENT_LENGTH + ": " + "0",

                              "-H", Constants.DISPATCHER_HEADER_CONTENT_TYPE + ": " + Constants.DISPATCHER_HEADER_CONTENT_TYPE_VALUE ,

                              dispatcher

                      };

                      try

                      {

                          ProcessBuilder process = new ProcessBuilder(command);

                          process.start();

                      }

                      catch (IOException e) {

                          log.error(e.getMessage(), e);

                      }

                  }

              }

      }

        • 1. Re: Can you help me work cache activation?
          PuzanovsP Experience Cloud MVP

          Dear Chung,

           

          Following piece of source code might be relevant for you[0]

           

          Chris Millar@Adobe has done stunning work at implementing Dispatcher functions, have a look, they are perfect and elegant.

           

          [0] slick-2/DispatcherServiceImpl.java at master · auniverseaway/slick-2 · GitHub

           

          Regards,

          Peter

          • 2. Re: Can you help me work cache activation?
            chung yonge16845482 Level 1

            Dear Peter

             

            the source code doesn't work in my local.

             

            I am trying to implement how to invalidate Dispatcher Cache from the Authoring Environment.

             

            When I publish page from the Authoring Environment, cache deletes successfully on dispatcher.

             

            But cache doesn't activate on dispatcher.

             

            Regards

            Chung Yong.

            • 3. Re: Can you help me work cache activation?
              PuzanovsP Experience Cloud MVP

              Dear Chung,

               

              My cincere apologies that the code did not work for you,

               

              Also, have a look at following article[0], also remember that AEM has already done all the hard work for you and they provide Replicator API for your needs[1].

               

              [0] Invalidating Cached Pages From AEM

              [1] Replicator ("The Adobe AEM Quickstart and Web Application.")

               

              Regards,

              Peter

              • 4. Re: Can you help me work cache activation?
                chung yonge16845482 Level 1

                Dear Peter

                 

                I have already read the information([0] Invalidating Cached Pages From AEM) last week.

                 

                I don't think there is the way to activate cache in information([1] Replicator ("The Adobe AEM Quickstart and Web Application."))

                 

                I can not understand why cache deletion works well and cache activation doesn't work on dispatcher in my source code.

                 

                Regards

                Chung Yong.

                • 5. Re: Can you help me work cache activation?
                  PuzanovsP Experience Cloud MVP

                  Dear Chung,

                   

                  Normally, from AEM -> Dispatcher, via the API documented above, we update .stat file to let Dispatcher know that the files lower down in the tree need to be re-rendered from the server defined to it as renderer.

                   

                  We do not Activate any content from Author to Dispatcher, we only tell Dispatcher, which files need to be updated from the Content Publishing Server.

                   

                  Happy to be corrected here through )

                   

                  Regards,

                  Peter

                  • 6. Re: Can you help me work cache activation?
                    chung yonge16845482 Level 1

                    Dear Peter

                     

                    But from AEM -> Dispatcher, via the API documented above I have already deleted any content from Author to Dispatcher.

                     

                    Can I delete any content from Author to Dispatcher only?

                    Can I not activate any content from Author to Dispatcher?

                     

                    Regards

                    Chung Yong.

                    • 7. Re: Can you help me work cache activation?
                      PuzanovsP Experience Cloud MVP

                      Dear Chung,

                       

                      Yes, you can only delete from Author to Dispatcher (as of current Dispatcher module version 4.3.1 and apache 2.4).

                       

                      You can't activate content e.g. POST data from Author to Dispatcher.

                       

                      Only Dispatcher can GET data from Author, after it receives notification that .stat file has updated or no content is cached.

                       

                      Some AEM customers solve activation by switching from Dispatcher to Varnish or CDN which provide functionality to prepare content for customer and thus avoid additional fetch request delay.

                       

                      Regards,

                      Peter

                      • 8. Re: Can you help me work cache activation?
                        chung yonge16845482 Level 1

                        Dear Peter

                         

                        I found the below one yesterday.

                         

                        Cannot re-cache files through invalidate.cache

                        https://forums.adobe.com/thread/1219628

                         

                        He had the same problem with me 5 years ago.

                        He had the same version Apache Server 2.2 on Windows with me.

                         

                        But He upgraded From Apache Server 2.2 to Apache Server 2.4.

                        He changed the operating system from Windows to Linux because Apache Server 2.4 for dispatcher didn't support windows.

                         

                        Download Dispatcher Web Server Modules

                        https://www.adobeaemcloud.com/content/companies/public/adobe/dispatcher/dispatcher.html

                         

                        And He solved the problem.

                         

                        I wonder if I have to change my operating system from Windows to Linux in order to install Apache Server 2.4 for dispatcher like him

                        • 9. Re: Can you help me work cache activation?
                          PuzanovsP Experience Cloud MVP

                          Dear Chung,

                           

                          You might be spot on here.

                           

                          Please test it and report to community whether this is the issue.

                           

                          Regards,

                          Peter

                          • 10. Re: Can you help me work cache activation?
                            chung yonge16845482 Level 1

                            Dear Peter

                             

                            I am sorry I want to find the solution on Windows.

                             

                            it is hard for me to test it on Linux because I am not familiar with Linux.

                            • 11. Re: Can you help me work cache activation?
                              smacdonald2008 Adobe Employee

                              FRom that older thread - solutions seems - "I was using the Windows version of Dispatcher (with Apache server) which is not as updated as the Linux version. We have re-tested the program in Linux environment with more updated version of Apache and Dispatcher and it seems that re-caching is working fine."

                              • 12. Re: Can you help me work cache activation?
                                chung yonge16845482 Level 1

                                Hello,

                                 

                                I misunderstood I deleted and activated cache from the author instances to dispatcher.

                                 

                                so far I deleted and activated cache from the publish instances to dispatcher.

                                 

                                I fixed the problem that doesn't create cache after cache deletion.

                                 

                                I applied sleep method between delete method and activate method like below that.

                                 

                                boolean deleted = invokeToDeleteCacheFromCurl(getDispatcherUris(resolver), replicationAction.getPaths(), ReplicationActionType.DELETE);

                                 

                                        boolean deleted = invokeToDeleteCacheFromCurl(getDispatcherUris(resolver), replicationAction.getPaths(), ReplicationActionType.DELETE);

                                                if(deleted) {

                                                    try

                                                    {

                                                        TimeUnit.SECONDS.sleep(1); // 1 second delay.

                                                    }

                                                    catch(InterruptedException e)

                                                    {

                                                        log.error(e.getMessage(), e);

                                                    }

                                                    invokeToCreateCacheFromGet(getDispatcherUris(resolver), replicationAction.getPaths(), ReplicationActionType.ACTIVATE);

                                                }

                                 

                                So it works well.

                                 

                                but I have another problem.

                                 

                                now cache creates only html files and it doesn't create folders(ex: test\_jcr_content\content\image.img.jpg) when I publish page in author environment.

                                 

                                now I am trying to fix the problem that doesn't create folders.

                                 

                                Here is my sources.

                                 

                                public static void execute(BundleContext bundleContext, Event event) {

                                        if (bundleContext != null && event != null) {

                                            final ServiceReference rrServiceReference = bundleContext.getServiceReference(ResourceResolverFactory.class.getName());

                                            final ResourceResolverFactory resolverFactory = (ResourceResolverFactory) bundleContext.getService(rrServiceReference);

                                            final ResourceResolver resolver = AdminService.getResourceResolver(resolverFactory);

                                            final ReplicationAction replicationAction = ReplicationEvent.fromEvent(event).getReplicationAction();

                                            final ReplicationActionType replicationActionType = replicationAction.getType();

                                            if (replicationActionType.equals(ReplicationActionType.ACTIVATE)) {

                                                boolean deleted = invokeToDeleteCacheFromCurl(getDispatcherUris(resolver), replicationAction.getPaths(), ReplicationActionType.DELETE);

                                                if(deleted) {

                                                    try

                                                    {

                                                        TimeUnit.SECONDS.sleep(1);

                                                    }

                                                    catch(InterruptedException e)

                                                    {

                                                        log.error(e.getMessage(), e);

                                                    }

                                                    invokeToCreateCacheFromGet(getDispatcherUris(resolver), replicationAction.getPaths(), ReplicationActionType.ACTIVATE);

                                                }

                                            } else if (replicationActionType.equals(ReplicationActionType.DEACTIVATE) ||

                                                    replicationActionType.equals(ReplicationActionType.DELETE)) {           

                                                invokeToDeleteCacheFromCurl(getDispatcherUris(resolver), replicationAction.getPaths(), ReplicationActionType.DELETE);

                                            }

                                        }

                                    }

                                 

                                 

                                private static void invokeToCreateCacheFromGet(List<String> dispatchers, String[] paths, ReplicationActionType methodType) {

                                        for (String path : paths) {

                                            for (String dispatcher : dispatchers) {

                                                HttpClient client = new HttpClient();

                                                if (!path.endsWith(".html") && !path.startsWith("/content/dam/")) {

                                                    path = path + ".html";

                                                }

                                                dispatcher = dispatcher.replace("/dispatcher/invalidate.cache", "") + path;

                                                GetMethod getMethod = new GetMethod(dispatcher);

                                                try {

                                                    client.executeMethod(getMethod);

                                                }

                                                catch (IOException e) {

                                                    log.error(e.getMessage(), e);

                                                }

                                                finally {

                                                    if (getMethod != null) {

                                                        getMethod.releaseConnection();

                                                    }

                                                }

                                            }

                                        }

                                  }

                                 

                                if I don't add below source codes in invokeToCreateCacheFromGet method, the cache doesn't even create html files.

                                 

                                if (!path.endsWith(".html") && !path.startsWith("/content/dam/")) {

                                                    path = path + ".html";

                                                }

                                                dispatcher = dispatcher.replace("/dispatcher/invalidate.cache", "") + path;

                                • 13. Re: Can you help me work cache activation?
                                  chung yonge16845482 Level 1

                                  Hello Everyone,

                                   

                                  I realized that I could call only html file(test.html) with HttpClient Get.

                                   

                                  Someone told me that I can call all file  paths(ex:folders and files) of test.html if I use WebClient.

                                   

                                  I have added in my source in order to use WebClient.

                                   

                                  \RCC-Solution-POC\pom.xml

                                      <dependency>

                                  <groupId>net.sourceforge.htmlunit</groupId>

                                  <artifactId>htmlunit</artifactId>

                                  <version>2.33</version>

                                              </dependency>

                                   

                                  \RCC-Solution-POC\bundle\pom.xml

                                   

                                          <dependency>

                                  <groupId>net.sourceforge.htmlunit</groupId>

                                  <artifactId>htmlunit</artifactId>

                                          </dependency>

                                   

                                  \RCC-Solution-POC\bundle\src\main\java\com\royalcaribbean\excalibur\solution\poc\core\cust omdispatcherflush\FlushOnPublish.java

                                   

                                  import com.gargoylesoftware.htmlunit.WebClient ;

                                  import com.gargoylesoftware.htmlunit.html.HtmlPage;

                                   

                                      private static void getTest2() {

                                          try {

                                              final WebClient webClient = new WebClient();

                                              final HtmlPage page = webClient.getPage("http://localhost-aem-dispatcher/content/rccexcaliburpoc/test.html");

                                              log.error("getTest2 : page : {}", page);

                                          }

                                          catch (IOException e) {

                                              log.error(e.getMessage(), e);

                                          }

                                      }

                                   

                                  But when I publish page  after I import WebClient and deploy it  using maven, I have the following error message.

                                   

                                  The package 'com.gargoylesoftware.htmlunit' is inside a non-bundle dependency less… (Ctrl+F1)

                                  Inspection info :

                                  This inspections reports usage of classes from packages not accessible inside the OSGi context - i.e. those  located in .jar files not packaged  as bundles, or not exported by bundles, or not imported  in manifest  file(if applicale). Asking  for such a classes may cause 'class not found' exceptions at runtime.

                                  The inspection is disabled in tests by default(see 'Check tests' option).

                                   

                                   

                                  1. 26.10.2018 10:26:49.591 *WARN* [Thread-15] org.apache.felix.eventadmin Service [com.royalcaribbean.excalibur.solution.poc.core.listeners.ReplicationListenerOnPublish,70 73, [java.lang.Runnable, org.osgi.service.event.EventHandler]] EventAdmin: Exception during event dispatch [org.osgi.service.event.Event [topic=com/adobe/granite/replication] {modifications=[{paths=[Ljava.lang.String;@359d9997, time=1540564008437, type=ACTIVATE, userId=, revision=null}]} | [java.lang.Runnable, org.osgi.service.event.EventHandler] | Bundle(royalcaribbean-excalibur-poc.core [520])] (java.lang.NoClassDefFoundError: com/gargoylesoftware/htmlunit/WebClient)
                                  2. java.lang.NoClassDefFoundError: com/gargoylesoftware/htmlunit/WebClient

                                          at com.royalcaribbean.excalibur.solution.poc.core.customdispatcherflush.FlushOnPublish.getTe st2(FlushOnPublish.java:370)

                                          at com.royalcaribbean.excalibur.solution.poc.core.customdispatcherflush.FlushOnPublish.execu te(FlushOnPublish.java:87)

                                          at com.royalcaribbean.excalibur.solution.poc.core.listeners.ReplicationListenerOnPublish.han dleEvent(ReplicationListenerOnPublish.java:31)

                                          at org.apache.felix.eventadmin.impl.handler.EventHandlerProxy.sendEvent(EventHandlerProxy.ja va:415)

                                          at org.apache.felix.eventadmin.impl.tasks.HandlerTask.runWithoutBlacklistTiming(HandlerTask. java:102)

                                          at org.apache.felix.eventadmin.impl.tasks.SyncDeliverTasks.execute(SyncDeliverTasks.java:104 )

                                          at org.apache.felix.eventadmin.impl.tasks.AsyncDeliverTasks$TaskExecuter.run(AsyncDeliverTas ks.java:166)

                                          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)

                                          at java.util.concurrent.FutureTask.run(FutureTask.java:266)

                                          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)

                                          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)

                                          at java.lang.Thread.run(Thread.java:748)

                                  Caused by: java.lang.ClassNotFoundException: com.gargoylesoftware.htmlunit.WebClient not found by royalcaribbean-excalibur-poc.core [520]

                                          at org.apache.felix.framework.BundleWiringImpl.findClassOrResourceByDelegation(BundleWiringI mpl.java:1574)

                                          at org.apache.felix.framework.BundleWiringImpl.access$400(BundleWiringImpl.java:79)

                                          at org.apache.felix.framework.BundleWiringImpl$BundleClassLoader.loadClass(BundleWiringImpl. java:2018)

                                          at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

                                          ... 12 common frames omitted

                                   

                                   

                                   

                                  I have no idea how  I can fix the error regarding WebClient.

                                   

                                  Regards

                                  Chung Yong.