Scheduler DB Tablespace was full the DB shutdown

Document ID : KB000092944
Last Modified Date : 11/06/2018
Show Technical Document Details
Issue:
Scheduler Oracle DB shutdown. Now getting 911 error when every jobs try to run even after a cold start.
Environment:
CA Product = WORKLOAD AUTOMATION DE
Product Code = DSWKLA99000
Product Release =11.3 SP3
Component Code = DSRIES
Cause:
After analyzing the logs we could see the below exception :

20180404 16:15:01.348 [essential] [INFO] Thread-25: New StatusMessage: [date: Wed Apr 04 16:15:01 EDT 2018, type: Log message, loggerName: relationaldatabase, level: ERROR, threadName: DM.Appl.MD_TPMS_DELETION_SCRIPTS_RT_QUEUE.1, message: SQL Exception for query: INSERT INTO ESP_APPLICATION (APPL_ID, JOB_NAME, APPL_NAME, APPL_GEN_NO, START_DATE_TIME, END_DATE_TIME, SCHEDULED_DATE_TIME, STATE, TAG, COMMENTS, EVENT_NAME) VALUES (?,?,?,?,?,?,?,?,?,?,?); the exception is: ORA-00001: unique constraint (DSERIES.SYS_C007144) violated 
, stackTrace: [java.sql.SQLException: ORA-00001: unique constraint (DSERIES.SYS_C007144) violated, , 
Resolution:
From the exception we could figure out that the problem is due to the duplicate keys which resulted in the unique key constraint exception. We maintain the individual tables max keys in the esp_long_keys which will be used to determine the next max key for a table. We could see there is some difference in the individual tables and the esp_longkeys, due to which it couldn't determine the key properly. After updating the max ids properly, the server started well.