Skip navigation

CQ5

Jan Amplexor
Jan Amplexor
Currently Being Moderated

How to clean tarJournal?

Aug 2, 2012 9:37 AM

Tags: #manager #cq5.5 #persistence #tar

Hi all,

 

We had a bug in which an endless loop was continuously creating and then deleting the same node.  I noticed that the size of the repository on disc (under crx-quickstart/repository) did grow significantly.  By running the tar optimization I was able to reduce the files under crx-quickstart/repository/workspaces/crx.default from about 30GB to 300MB.  However, under crx-quickstart/repository/tarJournal we still have data files amounting about 50GB, while I noticed on other servers this should also be around the 300MB.  The tar optimization process is not cleaning up this folder apparently.  How can this be cleaned up?

 

Thanks,

Jan

2340 Views   15 Replies   Latest reply: rudy.carranza, Sep 24, 2012 7:23 AM
 
Replies
  • Sham HC
    Currently Being Moderated
    1. Sham HC,
    Aug 4, 2012 6:09 PM   in reply to Jan Amplexor
    Report

    Follow [1] to identify  and narrow down the source of the repository growth.

     

    [1]   http://dev.day.com/content/kb/home/Crx/Troubleshooting/analyze-unusual -repository-growth.html

     
    |
    Mark as:
  • Jan Amplexor
    Currently Being Moderated
    2. Jan Amplexor,
    Aug 5, 2012 3:21 AM   in reply to Sham HC
    Report

    Thanks, but I know what caused the growth and the link you mentioned is indeed how I was able to confirm my suspicion.  I was able to clean up the repository, but now what do I do with this Tar Journal?  I don't find any documentation about how to clean it up.  Is it even possible?  The content of my Tar Journal is pretty useless right now, it just contains 50GB of create node x and delete it again... :/

     

    Regards,

    Jan

     
    |
    Mark as:
  • todd.haser
    Currently Being Moderated
    3. todd.haser,
    Aug 5, 2012 3:25 AM   in reply to Jan Amplexor
    Report

    Have you tried running the Data Store Garbage Collection?

    http://dev.day.com/content/kb/home/Crx/CrxSystemAdministration/DataSto reGarbageCollection.html

     

    Todd

     
    |
    Mark as:
  • Jan Amplexor
    Currently Being Moderated
    4. Jan Amplexor,
    Aug 5, 2012 4:14 AM   in reply to todd.haser
    Report

    No, but I don't see how the datastore garbage collection could help...  Unless of course it does more than clean up the datastore?

     

    Regards,

    Jan

     
    |
    Mark as:
  • todd.haser
    Currently Being Moderated
    5. todd.haser,
    Aug 5, 2012 4:20 AM   in reply to Jan Amplexor
    Report

    try checking disk usage to s

     

    Connected by MOTOBLUR™

     
    |
    Mark as:
  • Jan Amplexor
    Currently Being Moderated
    6. Jan Amplexor,
    Aug 5, 2012 4:37 AM   in reply to todd.haser
    Report

    I have run the disk usage report and that showed me that my repository was just 300MB instead of 30GB, so running the tar optimization fixed that, as already mentioned in my initial post.  My problem is with the tar journal, the transaction log if you will, not with the repository, not with the datastore...

     
    |
    Mark as:
  • Sham HC
    Currently Being Moderated
    7. Sham HC,
    Aug 5, 2012 6:31 AM   in reply to Jan Amplexor
    Report

    Before rotating I want to know the actual cause. Since we are clear on cause you can rotate it.

    Journal files that are older than the configured age are automatically deleted. The default is “P1M”, which means files older than one month are deleted.  Since issue happened for you recently It still exists and will be cleaned after a month.  In case you can't wait for a month,  If you are using default shared nothing configure in repository.xml the Journal tag with maximumAge.  The sytax for maximumAge refer [1].   In case you are using cluster then additional parameter maximumFiles need to be set. Take a look at [2].

     

    [1]  http://en.wikipedia.org/wiki/ISO_8601

     

    [2]   http://dev.day.com/content/kb/home/Crx/Troubleshooting/JournalTooMuchD iskSpace.html

     
    |
    Mark as:
  • Jan Amplexor
    Currently Being Moderated
    8. Jan Amplexor,
    Aug 6, 2012 9:23 AM   in reply to Sham HC
    Report

    Hi,

     

    The documentation you are referring to seems to be out of date for CQ 5.5.  I tried setting the maximumFiles parameter, but after restarting CQ wasn't working anymore.  For every request I get the following error in the logs:

        

    org.apache.sling.engine.impl.SlingHttpContext handleSecurity: AuthenticationSupport service missing. Cannot authenticate request. 

     

    The documentation you refer to is talking about a com.day.crx.core.journal.FileJournal while the Journal is now a com.day.crx.persistence.tar.TarJournal.  Do you know if there's an updated version of this documentation somewhere?

     

    Thanks,

    Jan

     
    |
    Mark as:
  • Sham HC
    Currently Being Moderated
    9. Sham HC,
    Aug 7, 2012 5:06 AM   in reply to Jan Amplexor
    Report

    Hi Jan,

     

    Updated the kb article please verify.

     

    Thanks,

    Sham

     
    |
    Mark as:
  • Jan Amplexor
    Currently Being Moderated
    10. Jan Amplexor,
    Aug 7, 2012 7:35 AM   in reply to Sham HC
    Report

    Thanks, that seems to be correct now.    Daycare also pointed out that all the possible configuration options are documented on [1], so maybe you can point to that as well on the kb article?

     

    Note that after changing the maximumAge from P1M to P3D (to test), I still had to run the Tar Optimization otherwise nothing happens.

     

    Regards,

    Jan

     

    [1] http://dev.day.com/docs/en/crx/current/administering/cluster.html#Clus ter Element

     
    |
    Mark as:
  • rudy.carranza
    Currently Being Moderated
    11. rudy.carranza,
    Sep 20, 2012 10:48 AM   in reply to Jan Amplexor
    Report

    I changed the maximumAge and run the tar optimization but there were no changes in the tarJournal folder :s

    Is there additional configuration necessary??

     
    |
    Mark as:
  • Jan Amplexor
    Currently Being Moderated
    12. Jan Amplexor,
    Sep 20, 2012 11:59 PM   in reply to rudy.carranza
    Report

    No, that should do the trick.  What version of CQ are you on?  The required configuration is different depending on the version.  For CQ5.5 you should follow the instructions in [1] under section "Rotate tarjournal in shared-nothing clustering" (supposing you use shared-nothing clustering).  More information about the configuration parameters can be found if you follow the link in the article ([2]).  However, if you're on CQ5.4 the configuration is different (5.4 is using a FileJournal while 5.5 is using a TarJournal).  Also, make sure that the Tar optimization process has actually finished completely.  If you let it run during the default scheduled times (somewhere between 2am and 5am I believe), then it might be possible that the optimization process was not yet completely finished at 5am, so maybe it was still working on the repository data files and did not yet begin with the tar journal files.

     

    Regards,

    Jan

     

    [1] http://dev.day.com/content/kb/home/Crx/Troubleshooting/JournalTooMuchD iskSpace.html

    [2] http://dev.day.com/docs/en/crx/current/administering/cluster.html#TarJ ournal

     
    |
    Mark as:
  • rudy.carranza
    Currently Being Moderated
    13. rudy.carranza,
    Sep 21, 2012 9:43 AM   in reply to Jan Amplexor
    Report

    I'm using CQ5.5 with the default configuration as a shared-nothing cluster. I did a few of changes in the file: crx-quickstart/repository/repository.xml to change the time of tar optimization. I changed:

    <Workspace name="${wsp.name}" simpleLocking="true">

         <PersistenceManager class="com.day.crx.persistence.tar.TarPersistenceManager">

              <param name="autoOptimizeAt" value="7:40-11:00"/>

         </PersistenceManager>

    </Workspace>

     

    <Versioning rootPath="${rep.home}/version">

         <PersistenceManager class="com.day.crx.persistence.tar.TarPersistenceManager">

              <param name="autoOptimizeAt" value="7:40-11:00"/>

         </PersistenceManager>

    </Versioning>

     

    and

     

    <Cluster syncDelay="5000">

         <Journal class="com.day.crx.persistence.tar.TarJournal">

              <param name="maximumAge" value="P1D"/>

              <param name="maxFileSize" value="256"/>

         </Journal>

    </Cluster>

     

     

    I also changed the file crx-quickstart/repository/workspaces/workspace.xml

    <Workspace name="crx.default" simpleLocking="true">

         <PersistenceManager class="com.day.crx.persistence.tar.TarPersistenceManager">

              <param name="autoOptimizeAt" value="7:40-11:00"/>

         </PersistenceManager>

    </Workspace>

     

    The tar optimization process has reduced the used space but the files under tarJournal are still there and I've noticed that when the optimization process a file optimizeNow.tar appears but not in the tarJournal folder. Am I missing something or doing something wrong?

     
    |
    Mark as:
  • Sham HC
    Currently Being Moderated
    14. Sham HC,
    Sep 23, 2012 5:21 PM   in reply to rudy.carranza
    Report

    Based on your configuration I do not see any issue. Might be records are withing one month & hence not reducing.

     

    To manually start tar optimization create a file optimize.tar under crx-quickstart/repository/tarJournal/

    If optimize.tar gets renamed to optimizeNow.tar by the system then you know it is running.

    Once it is completed the file will be deleted.

     
    |
    Mark as:
  • rudy.carranza
    Currently Being Moderated
    15. rudy.carranza,
    Sep 24, 2012 7:23 AM   in reply to Sham HC
    Report

    I was traying to delete the older files by placing a optimizeNow.tar file but it was deleted by the system. Placing a optimize.tar file instead did the job and deleted the older files. Thanks Sham.

     
    |
    Mark as:
Actions

More Like This

  • Retrieving data ...

Bookmarked By (0)

Answers + Points = Status

  • 10 points awarded for Correct Answers
  • 5 points awarded for Helpful Answers
  • 10,000+ points
  • 1,001-10,000 points
  • 501-1,000 points
  • 5-500 points