Errors seen in the catalina.out/stdout.log file halting processing of events on Spectrum Report Manager (SRM) server

Document ID : KB000048179
Last Modified Date : 14/02/2018
Show Technical Document Details

Description:

The following errors are seen in the catlina.out (Linux and Solaris) or stdout.log (Windows) file on the Spectrum Report Manager server:


     May 15, 2014 2:36:16 AM
     (SRM/AvailabilityHandler/bucketReader) (SRM_Avail) - (ERROR) - Critical failure with events; processing halted for all servers.
 
     May 15, 2014 2:36:16 AM (SRM/AvailabilityHandler/bucketReader) (SRM_Avail)
     - (ERROR) - insertOutages Caused by:
     org.springframework.dao.CannotAcquireLockException:
     PreparedStatementCallback; SQL [INSERT IGNORE INTO modeloutage (model_key,
     landscape_h, start_time, end_time, zero_length_outage,  outage_type,
     start_event_key, end_event_key) VALUES (?, ?, ?, ?, ?, ?, ?, ?)]; Lock wait
     timeout exceeded; try restarting transaction; nested exception is
     java.sql.SQLException: Lock wait timeout exceeded; try restarting transaction

Solution:

These errors occur when an archiving or purging is happening on one table and at the same time bucket processing needs that same table to be updated with new values.

The solution is to increase the innodb_lock_wait_timeout value in the $SPECROOT/mysqlmy-spectrum.cnf file and restart mysql and tomcat for the change to take affect.

  1. Shutdown tomcat
  2. Shutdown mysql
  3. Modify the $SPECROOT/mysqlmy-spectrum.cnf file. If the innodb_lock_wait_timeout parameter is not present, add it to the end of the file and set it to 300 as follows:
    innodb_lock_wait_timeout = 300
  4. Restart mysql
  5. Restart tomcat
  6. Monitor the catalina.out/stdout.log file for errors. If the errors reoccur after increasing the value of innodb_lock_wait_timeout, follow the above steps and increase to 600.

Keep increasing as needed until the error messages stop.