Spectrum Archive Manager not starting after system crash.

Document ID : KB000057378
Last Modified Date : 11/09/2018
Show Technical Document Details
Issue:

Spectrum Archive Manager not starting after system crash. The following error message is seen in the $SPECROOT/SSS/DDM/ARCHMGR.OUT file:

"Native error: 145, table './ddmdb/event' is marked as crashed and should be repaired" 

Environment:
Spectrum 9.x and up
Cause:

The system crashed caused the Archive Manager DDM database to become corrupt.

Resolution:

1. Run ArchMgr -repair to repair the database by doing the following:

  • Log into the system as the user that owns the Spectrum installation
  • If on Windows, open a bash shell by running "bash -login". If on Linux or Solaris, open a terminal window.
  • cd to the $SPECROOT/SS/DDM directory
  • Remove the .DDMDB.LOCK file if present and Archive Manager is not running
  • Run the following command: ./ArchMgr -repair
NOTE: Depending on the size of the DDM database, the repair could take a few minutes up to several hours (if DDM is many GB in size). If the repair is successful, start the Archive Manager with Spectrum Control Panel. To start Archive Manager via command line, see Additional Notes below. 

2. If the above repair was not successful, you can try ./ArchMgr -force -repair

3. If still not successful, you may need to perform some additional maintenance on the MySQL database
  • cd to <SPECROOT>/mysql/bin
  • run ./mysqlcheck -uroot -proot ddmdb
If errors are found, mysqlcheck will attempt to aut-repair them. If for some reason there is issue with the repair, you can try to Force Repair: 
  • ./mysqlcheck -f -r -uroot -proot ddmdb
You can also try running repair on the DB files individually: 
  • run ​./myisamchk -m data/ddmdb/*.MYI
  • if errors are found, run ./myisamchk -r data/ddmdb/*.MYI
NOTE: Starting in Spectrum 10.3 we are no longer using MyISAM MySQL Engine and have now moved to newer more efficient MySQL InnoDB Engine, thus this command will only work on 10.2.3 and lower.

Lastly, there is an option to set Archive Manager to repair automatically. Be Aware that for larger databases, this can cause significant Archive Manager startup delays, so use with caution.
  • add the following line to <SPECTRUM>/SS/DDM/.configrc: AUTO_REPAIR_DB=TRUE
  • save and restart Archive Manager from Spectrum Control Panel
Additional Information:
Start Archive Manager via command line: 
  • cd to <SPECROOT>/bin
  • run ./launchinstdbapp localhost ARCHMGR y ARCHMGR.OUT

If you also experienced a Spectrum SpectroSERVER crash and need instructions on how to load a backup database for recovery, please see this article: https://comm.support.ca.com/kb/spectrum-crashed-and-try-to-start-spectroserver-shows-error-database-was-not-gracefully-closed/KB000113595

For more information on repairing a corrupt DDM database, please reference the "DDM and Archive Manager Troubleshooting" section in the guides: https://docops.ca.com/ca-spectrum/10-3-0/en/administrating/database-management/ddm-database-maintenance/ddm-and-archive-manager-troubleshooting