GC Heap constantly huge in MOM

Document ID : KB000100322
Last Modified Date : 06/06/2018
Show Technical Document Details
Issue:
We're experiencing huge time spent on GC on MOM in quality environment. We increased the JVM Heap in order to improve this performance but was still getting the same issues. So we did a thread dump and It seems that we're getting a thread blocked that is causing this massive time (ms). The MOM log shows:

 [ERROR] [Harvest Engine Pooled Worker] [Manager] Unable to process data for Time: 1486692915000, Frequency: 15s, Count: 19
org.garret.perst.AssertionFailed: Assertion failed
at org.garret.perst.Assert.that(Assert.java:18)
at org.garret.perst.impl.PagePool.get(PagePool.java:232)
at org.garret.perst.impl.StorageImpl.loadStub(StorageImpl.java:2812)
at org.garret.perst.impl.StorageImpl.lookupObject(StorageImpl.java:2754)
at org.garret.perst.impl.LinkImpl.loadElem(LinkImpl.java:228)
at org.garret.perst.impl.LinkImpl.get(LinkImpl.java:21)
at org.garret.perst.impl.AltBtree$BtreePage.find(AltBtree.java:64)
at org.garret.perst.impl.AltBtree$BtreePage.find(AltBtree.java:70)
at org.garret.perst.impl.AltBtree$BtreePage.find(AltBtree.java:70)
at org.garret.perst.impl.AltBtree.get(AltBtree.java:850)
at org.garret.perst.impl.PersistentMapImpl.get(PersistentMapImpl.java:115)
at com.ca.apm.baseline.engine.core.PerstManager.get(PerstManager.java:119)
at com.ca.apm.baseline.engine.BaselineDataManager.loadOrCreateDeviationProfile(BaselineDataManager.java:403)

 
Environment:
APM 10.x
Cause:
This message indicate a problem updating the baseline database baselines.db (now called variance.db) and the most likely cause is corruption.


 
Resolution:
Start with a clean baseline database by following below steps:

- Shut down the EM
- Navigate to the <EM_Home>\data directory
- Look for the file named variance.db (or
baselines.db).
- Move it to another location.
- Restart the EM.
- The variance.db (or
baselines.db ) file will be re-generated.