Spectrum SCOM Connector is not working

Document ID : KB000072247
Last Modified Date : 27/02/2018
Show Technical Document Details
Issue:
We noticed that even though our Spectrum SCOM Connector  service is running, we are not seeing any alerts in Spectrum from SCOM. Looking at the alerts in the SCOM Manager, we noticed the alerts are not being forwarded to the registered Spectrum SCOM Connector.
Environment:
We are currently running Spectrum 10.1.1 and the Spectrum SCOM Connector is installed on our SCOM Manager server.
Cause:
By enabling the SCOM Connector Debug by adding the line "debuglevel = 3" in the SCOMConnector\.scomrc file and restarting the "Spectrum SCOM Connector" service revealed the following error in the SCOMConnector\CONNECTOR_DEBUG.OUT file:

2/23/2018 8:38:38 PM: SCOMSpectrumMgr::isConnected() - alarmDomain is available 
2/23/2018 8:38:38 PM: SCOMSpectrumMgr::isConnected() - callbackRegistered = 1, ss_switch = False, isPrimary = 1 
2/23/2018 8:38:38 PM: Getting new/updated alerts from SCOM... 
2/23/2018 8:38:38 PM: Received 500 alerts from SCOM 
2/23/2018 8:38:38 PM: SCOMWorker::run() - processing alert: https://scom.xxx.com:443/OperationsManager/default.aspx?DisplayMode=Pivot&AlertID=5e65d9c2-5351-4d2c-ae43-3dd8cffeb7fd, alarmID = , curResState = -1, alert->ResState = 255, alert->RepeatCount = 3 
2/23/2018 8:38:38 PM: Using PrincipalName: null 
2/23/2018 8:38:38 PM: Using CustomField1: 
2/23/2018 8:38:38 PM: clearing alarm on model 0x629b5906 
2/23/2018 8:38:38 PM: clearing alarm on model 0xaaa399a6 
2/23/2018 8:38:38 PM: FAILURE: System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. 
at CsCModelDomain.CsCModelPropList_var.{dtor}(CsCModelPropList_var* ) 
at ___CxxCallUnwindDtor(IntPtr pDtor, Void* pThis) 
at SCOMSpectrumMgr.clearAlarms(SCOMSpectrumMgr* , String modelName, String alertUrl) 
at SCOMWorker.run() 
2/23/2018 8:38:38 PM: Failed to process SCOM alert updates, waiting 30 seconds for services to become available... 

The System.AccessViolationException is a .Net exception that gets thrown when there is an attempt to read from or write to protected memory. If several read or write operations have occurred through bad pointers, memory may be corrupted. Sustaining Engineering reviewed the SCOM Connector code and determined this is likely the result of running an older SCOM Connector with a newer SpectroSERVER, and thus the conflict.

The SCOM Connector and SpectroSERVER must be the same release. Therefore, anytime you upgrade the SpectroSERVER, you must also upgrade the SCOM connector.
Resolution:
In our troubleshooting of the problem we found that the SCOM connector was in fact running an older release, 9.4.1. We found this by launching a File Explorer in Windows and navigating to the Install Directory of the SCOM Connector. In the File Explorer, we right clicked on the column header, and selected “More…”, and enabled “File Version” and “Product version”, which allowed us to see the installed and running connector was from the 9.4.1 release or Spectrum.
 
To resolve the issue:
  1. Stop the “Spectrum SCOM Connector” service from the Windows Services dialog window.
  2. Uninstalled the SCOM Connector by opening a command line and executing the following command from the SCOMConnector directory:  “SpectrumSCOMConnector.exe –remove”
  3. G to the SCOM Manager server and unregistered the SCOM Connector from the SCOM Manager by taking the following steps:
    1. Make a copy of the script found at https://blogs.technet.microsoft.com/kevinholman/2017/02/07/removing-unwanted-product-connectors-in-scom-2012-r2-and-scom-2016/
    2. Edit line 11 of the script to use the correct connectorName, and save the script as DeleteConnector.ps1, in the SCOMConnector directory
    3. Open a “PowerShell” and cd to the SCOMConnector directory
    4. Run the PowerShell script:  DeleteConnector.ps1
  4. Once the Script runs and the Connector is unregisterd and no longer seen in the SCOM Manager, rename the SCOMConnector directory.
  5. Go to the SpectroSERVER and bring over a copy of the $SPECROOT/SCOMConnector directory and place it on the SCOM Connector server.
  6. Make a copy of the scomrc.example file and name it .scomrc
  7. Make a copy of the corbarc.example file and name it .corbarc
  8. Edit the .scomrc file with the appropriate entries for your environment. You may want to look at the old .scomrc file as an example.
  9. Install the SCOM Connector service by running:  “SpectrumSCOMConnector.exe –install”. For more information see https://docops.ca.com/ca-spectrum/10-2-3/en/integrating/microsoft-mom-and-scom/install-the-scom-connector.
  10. Start the “Spectrum SCOM Connector” service
  11. Register and configure the SCOM Connector with the SCOM Manager. See https://docops.ca.com/ca-spectrum/10-2-3/en/integrating/microsoft-mom-and-scom/start-the-scom-connector/verify-that-the-scom-connector-is-running-properly.
Additional Information:
For more information regarding the SCOM Connector Integration, see the "Microsoft MOM and SCOM" documentation found at https://docops.ca.com/ca-spectrum/10-2-3/en/integrating/microsoft-mom-and-scom