How to set Jarvis kafka Logs Rotation

Document ID : KB000112510
Last Modified Date : 30/08/2018
Show Technical Document Details
Introduction:
An example of the process to log retention changes, this steps to advise the customer keeps the log size at 1 hour and 100 MB.
Background:
As disk space limited, customer requests to change the log retention in Kafka server.properties
Environment:
APM 10.7
Instructions:
An example to how process the log retention changes, this steps to advise the customer keeps the log size at 1 hour and 100 MB, and the users can change the values to the hours or bigger size according to their disk size.

1. Stop all java processes
2. Clear all data from the Data directory for Kafka (and zookeeper, if there is)
3. Clear data from /tmp directory (zookeeper keeps some reference data there)
4. Change the following setting in server.properties
a. Set log.flush.interval.ms=1000 - (that is 1 sec) The maximum amount of time a message can sit in a log before we force a flush
b. Set log.retention.check.interval.ms=60000 - (that is 1 sec) The interval at which log segments are checked to see if they can be deleted according to the retention policies
c. log.segment.bytes=100000000 ( that is 100 MB) The maximum size of a log segment file. When this size is reached a new log segment will be created.
d. log.retention.hours=1 ( that is 1 hour) The minimum age of a log file to be eligible for deletion (this is optional)
5. Start Kafka (and Zookeeper, if there is)
6. Start pumping data