Best Practice Guide for Logs collection and configuring there retention count

Document ID : KB000028442
Last Modified Date : 14/02/2018
Show Technical Document Details

Introduction:

In order to help us analyze, identify root cause and resolve the issue quickly, as possible there are some key information we need to collect from you. Below document list down lists of logs, thread dump, etc. we would like to get from the error facing instance.  We advise to collect the same during error and provide when opening a Support Ticket.

Background:

There are various logs being generated by our product, each having rolling up index and file size. Among all logs there are some important logs for which we recommend to retain a large retention number to prevent overwriting of content, as we had observed in case of an error the vital information gets overwritten and make it difficult to identifying root cause in first place.

Environment:

Applies to environment 4.7(all version), 5.0(all version), 5.5(all version), 6.0(all version), 6.1(all version)

Instruction:

The value for number of log files and their size is determined considering disk space available in an environment. A trade-off can be made between file-size and retention number.

  • Number for MaxBackupIndex(log retention no.) value should be set to a large number in large environments, especially for Management and Execution servers. 
    • For example: in environments with 100+ agents connected to a NES, MaxBackupIndex can be either set to 200 (size 5MB) or 100 (size 10MB).

 

  • We recommend to derive and evaluate most feasible numbers for file size and MaxBackupIndex to provide logs coverage of 24 hours duration

Important log files in respective component along with there location

ComponentFile LocationImportant Log FilesModification to increase log file number
Management Server\webapps\datamanagement\WEB-INF\log4j.properties<Install Directory>/logs/nolio_dm_all.loglog4j.appender.RegularDM.MaxBackupIndex=30
Execution Server\webapps\execution\WEB-INF\log4j.properties
  • <Install Directory>/logs/nimi.log
  • <Install Directory>/logs/Nolio_exec_all.log
  • <Install Directory>logs/execution.log
  • log4j.appender.RegularExecution.MaxBackupIndex=20
  • log4j.appender.ExecutionFileAppender.MaxBackupIndex=20
  • log4j.appender.NimiFileAppender.MaxBackupIndex=30
Agent\conf\log4j.properties
  • <Install Directory>/logs/nimi.log
  • <Install Directory>/logs/nolio_all.log
  • <Install Directory>/logs/nolio_action_exe.log
  • log4j.appender.Regular.MaxBackupIndex=15
  • log4j.appender.NimiFileAppender.MaxBackupIndex=30
  • log4j.appender.ActionsAppender.MaxBackupIndex=15
Retrieval/Repository Agents **\conf\log4j.properties
  • <Install Directory>/logs/nimi.log
  • <Install Directory>/logs/nolio_all.log
  • <Install Directory>/logs/nolio_action_exe.log
  • log4j.appender.Regular.MaxBackupIndex=20
  • log4j.appender.NimiFileAppender.MaxBackupIndex=20
  • log4j.appender.ActionsAppender.MaxBackupIndex=30

 ** Retrieval or Repository agents, in product terminology are referring to agents which are responsible for fulfilling request to download and distribute artifacts.

When and how to collect log files?

  •  All the below information need to be collected when issue is seen most preferably prior to restart of NAC or any other services
    ComponentGood to have
    Management ServerAll logs
    Execution Server(If multiple Execution servers we need logs from all impacted)All logs
    Agent (If multiple Agents we need logs from all impacted agents)All logs
    UIAll logs

 

  • Collection and sharing of specific log file is quick and can help in most of cases, but in some scenario we need whole logs set to root cause an issue. So as a best practice we recommend to collect all logs from component before it get overwritten.
  • We always recommend to collect logs via out of box product feature to collect logs. For quick recall collecting logs via ASAP: Go to Administration tab-> Agent management console ->right click on specific agent -> Collect logs. (feature of log collection is available to my best from 4.1 version onward).  Refer to KB on how to do it!
  • In some cases where customer maintain large number of log retention we request them to manually collect the logs, as this utility will zip all logs folder irrespective of time-frame. In such scenario it can cause an overhead in collecting logs in GB's.

 

Additional Information:

Beside logs sometimes we need additional information from JMX console of product. Please find details of where you can collect those information.

 

ComponentJMX Location (localhost need to be replaced with NAC/NES/NAG IP/Hostname)JMX Operations
Management Serverhttp://localhost:20203/mbean?objectname=noliocenter%3Atype%3Dinfo

Execute below operation and capture the result of below operations

   a. requestStackTrace

  b. listActiveRuns

   c. memoryStatus

Execution Serverhttp://localhost:20203/mbean?objectname=com.nolio.nimi.jmx%3Aname%3DnimiJMX%2Ctype%3DNimiJMX

Execute below operation and capture the result of below operations

   a. memoryDump

   b. threadDump

Agenthttp://localhost:8282/mbean?objectname=Deployer%3Acommunication%3DnimiManager

Execute below operation and capture the result of below operations

   a. threadDump

   b. retrieveEnginesStatus

   c. memoryDump

Hibernate SQL Logginghttp://localhost:20203/mbean?objectname=Logging%3Atype%3DLog4j

Execute below operation to turn on/off hibernate SQL logging

   a. turnOnHibernateSQLLogging

   b. turnOffHibernateSQLLogging

HTTP Request Logging http://localhost:20203/mbean?objectname=Logging%3Atype%3DLog4j

Execute below operation to turn on/off hibernate SQL logging

   a. turnOnHttpRequestsLogging

   b. turnOffHttpRequestsLogging

 

Footnotes 

  • Release Automation formerly known as Nolio
  • Data Management Server formerly known as Nolio Automation Center (NAC/DMS)
  • Execution Server formerly known as Nolio Execution Server (NES/ES)
  • Agent/s formerly known as Nolio Agent (NAG/Agent)