Secondary hub monitoring with heartbeat alarms

Document ID : KB000010717
Last Modified Date : 14/02/2018
Show Technical Document Details
Introduction:

Some customers want additional notification if a secondary hub and it's nas stop communicating with the primary hub.  The following configuration will create hearbeat alarms every 2 minutes from the secondary hub and monitor them every 5 minutes from the primary.  If these informational heartbeat alarms stop arriving, a warning alarm will be generated.
The high-level procedure is to configure the alarm monitoring on the primary hub and then configure a script on the secondary hub to send informational heartbeat alarms to the primary.
**This assumes the nas probe is installed on both the primary and secondary hubs.**

Background:

**These scripts are provided for informational use only.  CA Support does not helpwith script writing or configuration**

Instructions:

On the Primary hub:
1. Add triggers to the primary hub nas.   
Add one trigger for each secondary hub to be monitored.  To eliminate confusion, we strongly recommend that these triggers be have the same name as the secondary hubs that you will be monitoring.  To create an appropriate trigger:
a. Click the Trigger tab, right-click white space and select New.
b. Add "/.*Heartbeat.*/" (without quotes) to the Message String.
c. Click OK and then give the trigger the name of the secondary hub you want to monitor.

 

2. Create a script for the primary hub nas to look for triggers from specific hubs.
The script checks the trigger to see if the alarm exists and then clears any existing alarms.  If there is nothing in the trigger, this script will send an alarm.  The names in this script are the names of the triggers that are configured in the nas Triggers tab.
Use the file "primary_nas_monitor_heartbeats" as a template.  Edit this to have the script look for the triggers you created in step 1.

-- Script to evaluate whether a remote nas heartbeat alarm has been replicated up to the primary.
--
-- Create a trigger in the primary nas and change entries like "<trigger_name 1>" below to the name of your trigger(s).

function ClearTriggerAlarms(trigger_name)
   msgids=""
   t = trigger.alarms(trigger_name)

   -- iterate through triggers alarms (if any) and add to msgids
   for i,a in pairs(t) do
      msgids=msgids .. a.nimid .. ","
   end

   if msgids ~= "" then
      action.close (msgids)
   end
end

--  This is where to substitute <..> for the correct trigger name.
triggers={"<trigger_name 1>","<trigger_name 2>", "<trigger_name 3>"}

for i,tr in pairs(triggers) do
   print(tr)
   if trigger.state (tr) then
      print("Heartbeat message from "..tr.." found. Resetting trigger")
      nimbus.alarm(0,"Heartbeat received from "..tr..". All OK",tr)
      ClearTriggerAlarms(tr)
   else
      print("Missing heartbeat message from "..tr..". Generate alarm")
      nimbus.alarm (4,"Missed heartbeat from downstream nas on "..tr..". Possible replication issue",tr)
   end
end

 

3. Use the primary nas Scheduler tab to run the script intermittently.  Use the Mode "By Recurring Event", Operation "Execute Script", run it every 5 minutes and select the "primary_nas_monitor_heartbeats" script.

 

 

On one or more secondary hubs:
1. On the secondary hub, make an alarm creation script and save it into /nimsoft/bin directory
The text of a Windows script file:
   @echo off
   rem Change the "<secondary hub name>" to the name of your secondary hub.
   "C:\Program Files (x86)\Nimsoft\bin\nimalarm.exe" -l 1 -S <secondary hub name> HeartbeatAlarm

A the text of a Linux script file:
   # Change the "<secondary hub name>" to the name of your secondary hub.
   ./nimalarm -l 1 -S <scondary hub name> HeartbeatAlarm


2. Configure a shell script or cron job to repeat this alarm creation script.
In Linux:
a.Make this script executable with:   chmod 755 <script_name>
b. Then schedule the script to repeat with:  crontab -e
   Add this text to the cron job:  2 * * * * /opt/nimsoft/bin/nas_heartbeat_alarm