duplicate model alarms in Spectrum's Virtual Host Manager (VHM).

Document ID : KB000008620
Last Modified Date : 14/02/2018
Show Technical Document Details
Issue:

We are we seeing duplicate model alarms, in Spectrum for Virtual Host Manager (VHM) models, with systemEdge agents?

e.g. Duplicate model detected. or Duplicate IP with different MAC detected.

We have ruled out the documented known issues - see additional information below.

Environment:
This problem can be seen at any version.In this example we are using:vCenter 5.5.VC AIM and SystenEdge are 5.9.0.Spectrum 10.2.1
Cause:

There can be various reasons for this, which are documented by CA - see below for details.
If the documented solutions do not work, the cause may be outside of the Spectrum application. 
If we consider the other elements that are required to discover virtual models in VHM, then we should also look at the SNMP agent, providing the virtual functionality. 

Is it possible to get an SNMP walk of the agent? 

In this example, a SNMP walk of the SystemEdge agent with the Sapwalk application, showed the agent had an SNMP problem, with the following error.

#Agent is broken. Getnext returned request oid. Trying 1.3.6.1.2.1.6.19.1.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.5250.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1
#ERROR:LEXORDER:1.3.6.1.2.1.6.19.1.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.80.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0, Integer , 0
#ERROR:LEXORDER:1.3.6.1.2.1.6.19.1.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.135.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0, Integer , 0
#ERROR:LEXORDER:1.3.6.1.2.1.6.19.1.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.445.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0, Integer , 0
#ERROR:LEXORDER:1.3.6.1.2.1.6.19.1.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2638.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0, Integer , 0
#ERROR:LEXORDER:1.3.6.1.2.1.6.19.1.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.3389.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0, Integer , 0
#ERROR:LEXORDER:1.3.6.1.2.1.6.19.1.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.4105.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0, Integer , 0
#ERROR:LEXORDER:1.3.6.1.2.1.6.19.1.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.4728.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0, Integer , 0
#Agent is broken. Current walk stopped to prevent looping. Trying 1.3.6.1.2.1.6.19.1.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.4105.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.1
#ERROR:LEXORDER:1.3.6.1.2.1.6.19.1.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.80.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0, Integer , 0
#ERROR: Walk ended to prevent continued looping in broken lexicographical ordering. Max allowable lexi errors exceeded.

To rule out a SapWalk problem, WalkTree was also used, to walk the mib which also got stuck in a loop at OID:
1.3.6.1.2.1.6.19.1.1.0.0.0.0.0.0.0.0.0.0.0.5666.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0

Reinstalling the agent gave the same result.

 

Where can we get further information:
If the port161 (or port1691) directory from the same host, that the walk was generated against, contains logs.

If they do not provide a root cause, then debug data, can be generated, by adding the following lines, in the /port161/sysedge.cf file.

sysedge_logfile sysedge.log 20000 6
sysedge_loglevel debug3

Save the file and restart the SystemEdge services.
After the problem reoccurrs, the /port1691/sysedge.log files can be analysed.

We them ran a netstat of the host where the SE agent is installed, which showed the Nagios system agent is also installed and listening on port 5666.

This was the mib entry causing the problem.

 

Resolution:

Change the port that the Nagious system agent uses.

This now allows us to walk the mib and no new duplicates get created in Virtual Host Manager (VHM).

Additional Information: