When is a group actually deleted from a distributed cluster or remote polling environment?

Document ID : KB000023690
Last Modified Date : 14/02/2018
Show Technical Document Details

Question:

When is a group actually deleted from a distributed cluster or remote polling environment?

Environment:

eHealth (all supported versions)
All supported plattforms

Answers:

What does the term "soft delete" mean in reference to eHealth groups?

"Soft Delete" is a function that was added into eHealth with the advent of distributed eHealth clusters. When a group is deleted, it isn't removed from the database right away. Instead a value "expire_time", reflecting the current system time,  is set on the group. At the point in time that the group is "soft deleted" it is no longer available through eHealth for tasks such as scheduled reports.

A scheduled eHealth job named "FSA Scrubber" runs every four hours. One of the tasks that  FSA Scrubber performs is to evaluate groups to see if they have been soft deleted and their expire_time has been set. It then sums "expire_time" and an eHealth environment variable named NH_DB_DELETED_OBJECTS_TIME2KEEP, which defaults to 168 hours (1 week). If the sum of these two values is less than the current system time, the group is then "hard deleted", completely removed from the eHealth system.  In summary, groups are soft deleted when they are originally deleted. They are then hard deleted when the scheduled FSA Scrubber job runs and one week has passed since the original group deletion.

 

What was "soft delete" created to address in eHealth?

Soft Delete was added to eHealth to address a potential issue with eHealth groups in a distributed cluster.

In a Distributed eHealth cluster, group information is replicated in real time across all cluster members. The new group is automatically replicated to all other eHealth cluster members. The same is true of a group deletion, with a group deletion actually signaled as a soft delete. This allows the group to maintain it's original group_id value, which eHealth uses in place of the group name in tasks such as scheduled reports.

If the soft delete function did not exist, the results of the situation above would be that the then a group would be completely removed from the eHealth cluster member systems that were in communication. The group would still be available and able to be modified on any system that was out of communication with the cluster. It could then be modified (by adding group members, for example) on this out-of-communication system.  When the system regained communication with the cluster, the modified group would be propagated to the cluster and the cluster would now see this group with an incomplete set of group members, since it was hard deleted and its original members created on all cluster members except this system were deleted/lost.  This would mean that any reports scheduled for this group would now be invalid, as they now won't report on the intended elements. It would alternately be possible that when the out-of-communication system regains communication with the cluster, the deletion from the cluster would be propagated to the system that regained communication and the modified group would then be deleted altogether.

 

What issues may be raised by the soft delete function?

The most common issue is seen when a group is deleted and an attempt is made to recreate the group from the eHealth console before its expire time has passed and the group has been hard deleted by the FSA Scrubber scheduled job. In this case, a message saying that the group cannot be created because it is currently marked for deletion will be displayed and the group will not be created.  However, there is a way to hard delete a group once it has been soft deleted. The command to accomplish this is nhScubFsa.  The proper syntax to use this command is as follows.

nhScrubFsa  -deleteObjectHours 0

 

Potential issues with using "nhScrubFsa -deletedObjectHours 0"

By using this command you have removed the safeguard that soft delete was created to address, and it is possible that you could have groups recreated with new group id's, causing issues such as scheduled report failures or reports that contain only a subset of the intended elements.

Another problem arises when this command is used in an eHealth remote poller environment.  For example, a group is created on a remote poller. This group is then created on the central poller automatically when the central picks up the configuration change from the remote poller, typically on the next import poll. The same group is then deleted on the remote poller (actually soft deleted, because this is how all group deletions function). This group is then soft deleted on the central poller automatically when it picks up the configuration change from the remote. The command "nhScrubFsa -deletedObjectHours 0" is then run on the remote poller, which hard deletes the group. Since this is not technically a configuration change, no notification is made to the central poller of this event.

A group with the same name is now created on the remote poller.  This group will not have the same group_id as the old group, since id's cannot be reused in eHealth. An attempt is then made to create this group on the central poller automatically when it picks up the configuration change from the remote poller. However, because this group still exists on the central poller in a soft deleted state the attempt will fail and the group will not be created. No errors will be written to any logs regarding this failure, but the effect will be that a group which now exists on the remote poller will not exist on the central poller. The only way to bring groups back into synchronization is through the use of the nhRpFindDiffs command. Alternately, the command "nhScrubFsa -deletedObjectHours 0" could be run on the central poller after the group was soft deleted from both servers. To know when the group is soft deleted from the central poller, you must have an understanding of eHealth remote polling functionality, or you must check that the group is no longer available for editing or for scheduled reports on the central poller.

 

Why recreate a group with the same name as a previously existing group?

Creating a group with the same name as a previous group does not preserve group_id. As all scheduled jobs for groups currently use group_id in place of group name, doing this will effectively strand the scheduled reports with no valid group_id to run against, as the group_id that they were originally scheduled for has been deleted. There is no problem with doing this, but any jobs that you intended to run against this group will have to be rescheduled. If the intention is to merely create a new group with the same name as a previously deleted group and preservation of group_id is not an issue, it is easier to simply rename the original group. 

 

What can  be done to avoid potential group issues surrounding soft deletion?

The easiest way to avoid problems with soft deletion is to refrain from the use of the "nhScrubFsa -deletedObjectHours 0" command and allow groups to become hard deleted by the combination of the scheduled FSA Scrubber job and the environment variable "NH_DB_DELETED_OBJECTS_TIME2KEEP".

In a standalone eHealth environment, the only problem with deleting and recreating groups with the same name is that the new group will have a new group_id, so it will not automatically replace a group in a previously scheduled eHealth report, for example. This is a necessary limitation as group_id's are used to uniquely identify groups and cannot be reused.