CA PPM: Multicast testing to confirm active nodes in cluster

Document ID : KB000051927
Last Modified Date : 30/08/2018
Show Technical Document Details
Issue:

The following are symptoms of multicast not working in CA PPM:

1. <server_url>/niku/nu#action:security.logs does not show all servers in the cluster

2. PPM command: admin tower
does not list all clients/services

3. CSA server dropdown does not show all nodes in the cluster

4. Jobs will error out with: ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired

5. Processes not starting or completing

 

 

 

Cause:
Multicast is not working.
Resolution:

For affected environment, please have the infrastructure team run the following command on all nodes on the PPM cluster:

admin general enable-discovery

Additional Information:

To test multicast communication using Jgroups:

Jgroups is the open source component that handles multicast communication between Clarity servers in a Cluster. The following procedure assumes you have at least two clarity servers and the corresponding network configuration have been made to allow multicast traffic.

On the machine running NSA (SENDER) execute the following:

For UNIX:

export CLASSPATH=$NIKU_HOME/lib/jgroups-all.jar

For Windows:
set CLASSPATH=%NIKU_HOME%/lib/jgroups-all.jar

Then execute:

java org.jgroups.tests.McastSenderTest -mcast_addr the_multicast_address -port the_beacon port -use_all_interfaces
 
e.g. 

java org.jgroups.tests.McastSenderTest -mcast_addr 2XX.xx.xx.x -port 9090 -use_all_interfaces -ttl 32


B. On the application servers (RECEIVER):

For UNIX:

export CLASSPATH=$NIKU_HOME/lib/jgroups-all.jar
 
For Windows:

set CLASSPATH=%NIKU_HOME%/lib/jgroups-all.jar
 
Then execute:

java org.jgroups.tests.McastReceiverTest -mcast_addr the_multicast_address -port the_beacon port -use_all_interfaces

e.g.

java org.jgroups.tests.McastReceiverTest -mcast_addr 2xx.xx.xx.x -port 9090 -use_all_interfaces


The same multicast address must be used on both the SENDER and on the RECEIVER.
You should use the same multicast address/port specified in the CSA.
If this fails, try using a different address/port.

Once the sender and receiver are running, messages from SENDER to RECEIVER
should been visible in the terminal windows.

The same message send from the SENDER window should IMMEDIATELY appear in the
RECEIVER window.

This test should be done on each application server to see if it can RECEIVE messages from the SENDER.

The best way to test is to type in:

If no information is being passed to the RECEIVER, then the environment is not
passing multicast traffic.

Contact the site network team to troubleshoot further as there could be more than one of the root causes:
1. Lack of IGMP Snooping enabled on the switch in between the app servers
2. A server patch/change