Use of the SmartStor merge tool on APM 10.5.x data can potentially corrupt SmartStor data.

Document ID : KB000008270
Last Modified Date : 14/02/2018
Show Technical Document Details
Issue:

Seeing in the APM 10.5.x EM logs many messages such as 

The log is filled with messages like: 

8/30/17 09:32:45.775 PM EST [WARN] [Smartstor/Superstor Spool] [Manager] Metric key id 0x100000029 illegally used by two metrics: [SuperDomain|Custom Metric Host (Virtual)|Custom Metric Process (Virtual)|Custom Metric Agent (Virtual) (Collector2)Enterprise Manager|Configuration:Number of Agent Clusters] and [SuperDomain|Custom Metric Host (Virtual)|Custom Metric Process (Virtual)|Custom Metric Agent (Virtual) (Collector2)Alerts|Default:Backend Errors]; ignoring second. 

 

Environment:
Upgrading to APM 10.5.x
Cause:

This may be due to the Smartstor merge tool that is included in APM 10.5, corrupting the Smartstor metadata.

Resolution:

The recommendation from Engineering is try avoid using the merge tool if possible. If it already occurred, use the new SmartStor repair tool added in 10.5.1HF#39 or 10.5.2SP2 to repair the corrupted data. (repair_metadata, repair_data).

 

Additional Information:

The same error can also happen with duplicate management modules -- After upgrade from 10.3 to 10.5 the MOM log is full of WARNing messages "Metric key id 0xnnnnnnnn illegally used by two metrics ...." which are repeated every 15 seconds  TEC1455096