My NFA Harvester Server is running out of Disk space.

Document ID : KB000046447
Last Modified Date : 22/05/2018
Show Technical Document Details
Introduction:

My NFA Harvester is running out of available Disk Space and I am afraid the drive will fill up. 

Background:

The NFA Harvester is responsible for processing and storing a high volume of data.

In a 2 tier environment a Harvester is responsible for storing three types of data:

1. Flow Forensics Data (Stored for up to 24-48 hours) stored in the \<Install Dir>\Netflow\datafiles\HarvesterArchive\ directory.

2. 1 Minute Resolution Data (Stored for up to 30 days) stored in the \<Install Dir>\Netflow\datafiles\ReaperArchive\ directory.

3. 15 Minute Resolution Data (Some of which is stored for up to 13 months) stored in the \<Install Dir>\Netflow\datafiles\ReaperArchive15\ directory.

Data is pruned and maintained in two ways by the "CA NFA Data Retention" service:

1. Daily - This data types mentioned above are pruned automatically back to its preset limits daily if the server has sufficient space by the "CA NFA Data Retention" service.

2. Threshold Based -The "CA NFA Data Retention" service also will always try to ensure that you have at least 10% free disk space (5% in earlier versions) at all times.  The "CA NFA Data Retention" service will check the server once a minute to see if the 10% free disk space threshold has been violated.  If so it will prune data back, in order to maintain the 10% free disk space.

 

There are two situations in which Data Pruning may fail and may cause you to run out of disk space.

1. If the "CA NFA Data Retention" service is down data pruning will stop.  

2. If the "CA NFA Reaper" service is down, files will start to build up in the \<Install Dir>\Netflow\datafiles\ReaperInput and \<Install Dir>\Netflow\datafiles\ReaperWork\ directories being that data cannot be processed.  

Being that the "CA NFA Data Retention" service does not prune the ReaperInput/ReaperWork directories, the Data retention service will keep pruning the directories that it can and eventually prune all of your data.  Eventually that will cause the Data Retention Service to stop, and the the ReaperInput\ReaperWork directories to fill up the rest of the drive.

 

Environment:
NFA 9.1 and later
Instructions:

1. Ensure that your "CA NFA Data Retention" service and the "CA NFA Reaper Service" are running at all times.

2. Ensure you have configured your Harvester with the correct amount of Storage according to the NFA System Requirements to ensure you have enough storage for the amount of flows and interfaces you are monitoring with each Harvester.