May receive the following error: "The transaction log for database '<database name>' is full due to 'LOG_BACKUP'" when using SQL Server as database.

Document ID : KB000045922
Last Modified Date : 14/02/2018
Show Technical Document Details

Issue:

When utilizing Nolio + SQL Server, Nolio server will not start, with error messages in nolio_dm_all.log similar to the one in the title referencing a full transaction log.

 

Environment:

Any configuration that utilizes Microsoft SQL Server with the default(or smaller) transaction log size.

 

Cause:

Basically any process design that utilizes excessive WHILE loops waiting for specific events to occur, or just generates a lot of events during execution(an unusually large amount) and will run for extended periods of time, intentional or not, are going to present the biggest risk to this rarely encountered issue.  With every action that is executed during a deployment for example,  several 'steps' or 'step events' are generated in the database for auditing purposes, and these can add up quick.  Almost 100% of auditing data is stored in what is referred to as the 'offline' tables, the only data that is not stored there is the auditing data for things that are currently running, or have been running in the last 30 minutes, since this is the interval at which this data is 'purged' and/or moved to offline tables in a single DB transaction.  Typically this is never a problem, however if we have generated enough step events(5GB+ to be exact), most likely by accident, we will hit our issue.  In the logs you will see the following reference to the Microsoft SQL Server transaction log being full:

 

2016-02-09 13:37:03,805 [JobToOffline-1] ERROR (org.hibernate.engine.jdbc.spi.SqlExceptionHelper:144) - The transaction log for database 'ReleaseAutomation' is full due to 'LOG_BACKUP'.
2016-02-09 13:37:03,808 [JobToOffline-1] ERROR (com.nolio.platform.server.dataservices.services.hibernate.NolioHibernateTemplate:203) - Caught hibernate exception.
org.hibernate.exception.SQLGrammarException: The transaction log for database 'ReleaseAutomation' is full due to 'LOG_BACKUP'.

 

This will then be followed by a non-responsive management server in most cases, in which a restart will be of no relief. 

 

Resolution:

The only way to relieve this situation is to manually purge the millions of rows of auditing data from the online tables.  The two tables that will need to have their rows deleted are:

1. steps

2. step_events

After this task is completed, normal operations will resume on the management server with no additional harm done, thankfully there is no critical data ever stored within either of these tables.  One thing to note, the data purged from these 2 tables will cause the loss of the auditing data for the particular deployment in question, if this *must* be saved, we recommend dumping the table data from both to files that can be then imported to the offline_step_events and offline_steps tables respectively.