SpectroSERVER continuously running very high (100%) CPU

Document ID : KB000044282
Last Modified Date : 14/02/2018
Show Technical Document Details

ISSUE:  SpectroSERVER continuously running very high (100%) CPU and losing connection with OneClick. 

 

CAUSE:  High number of calls to the 15IHRebootMonitor inference handler for Host_systemEDGE models. The more Host_systemEDGE models in the database the more of a performance impact.

 

The following is the result of an analysis of the Incident.Data* file run on a SpectroSERVER where this issue has been seen. Notice the values highlighted in the "# of Calls" column.

Trigger ModHandl (Model Name) MTHandl (MType Name) AttrId (Attr Name) Inference Handler Name # of Calls User+Kernel Time(ms) Elapsed Time(ms) Memory
AttrChange 0xca67bf8 xxx 0x1160088 (Host_systemEDGE) 0x1164142 (hostSystemUpTime) 15IHRebootMonitor82652.332113.5010
AttrChange 0xca67b1d xxx 0x1160088 (Host_systemEDGE) 0x1164142 (hostSystemUpTime) 15IHRebootMonitor82522.223113.150
AttrChange 0xcaa2807 xxx 0x1160088 (Host_systemEDGE) 0x1164142 (hostSystemUpTime) 15IHRebootMonitor70032.044135.9180
AttrChange 0xca67a42 xxx 0x1160088 (Host_systemEDGE) 0x1164142 (hostSystemUpTime) 15IHRebootMonitor70001.95136.5460
AttrChange 0xca67afc xxx 0x1160088 (Host_systemEDGE) 0x1164142 (hostSystemUpTime) 15IHRebootMonitor69971.888135.6680
AttrChange 0xcaa2821 xxx 0x1160088 (Host_systemEDGE) 0x1164142 (hostSystemUpTime) 15IHRebootMonitor69622.004137.8540
AttrChange 0xcadb065 xxx 0x1160088 (Host_systemEDGE) 0x1164142 (hostSystemUpTime) 15IHRebootMonitor60881.699153.8150
AttrChange 0xcadf95e xxx 0x1160088 (Host_systemEDGE) 0x1164142 (hostSystemUpTime) 15IHRebootMonitor60881.706154.0160

 *Incident.Data files are created by doing the following on the SpectroSERVER system where the issue is seen:

1. Log into the SpectroSERVER system as the user that owns the Spectrum installation

2. Find the value of the SpectroSERVER process id

3. On a Windows system:

    a. Start a bash shell by running "bash -login"

    b. cd to the $SPECROOT/lib/SDPM directory

    c. Enter the following command where <PID> is the SpectroSERVER process id:

       ./kill.exe -USR1 <PID>

     d. Wait 15-20 minutes and re-enter the same kill command entered in step 3c

     e. Open a case and send the $SPECROOT/SS/support/Incident.Data file to CA Spectrum Support for analysis

4. On Linux and Solaris systems:

    a. Run the following command where <PID> is the SpectroSERVER process id:

       kill -USR1 <PID>

    b. Wait 15-20 minutes and re-enter the same kill command entered in step 4a

    c. Open a case and send the $SPECROOT/SS/support/Incident.Data file to CA Spectrum Support for analysis

 

RESOLUTION: This issue has been addressed in the following patches. Please contact CA Spectrum Support:

For Spectrum 10.01.00.00 the 10.01.00.D54 patch

For Spectrum 10.01.01.00 the 10.01.01.D53 patch

This fix is also scheduled to be included in Spectrum 10.02.00.00. No specific release date at the time of the publication of this knowledge document.