CA PPM: Multicast testing to confirm active nodes in cluster

Document ID : KB000051927
Last Modified Date : 06/07/2018
Show Technical Document Details
Issue:

Symptoms of multicast not working in CA PPM:

  • <server>/niku/nu#action:security.logs does not show all servers in the cluster
  • Rate Matrix Extraction job fails with error: ORA-20000: Error in NBI_EXTRACT_SP - : ORA-20000: Error in NBI_CLEAN_DATAMART_SP: ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired
  • Datamart Extraction job fails with error: NBI_CLEAN_DATAMART_SP: ORA-00054: resource busy and acquire with NOWAIT specified or timeout expired ORA-06512: at "SJC111P.NBI_EXTRACT_SP", line 362
  • Process-related issues
  • PPM command: admin tower, does not list all clients/services

 

Cause:
Multicast is not working.
Resolution:

For affected environment, please have the infrastructure team run the following command on all nodes on the PPM cluster:

admin general enable-discovery

Additional Information:

To test multicast communication using Jgroups:

Jgroups is the open source component that handles multicast communication between Clarity servers in a Cluster. The following procedure assumes you have at least two clarity servers and the corresponding network configuration have been made to allow multicast traffic. Please refer to the Clarity Installation guide (Sixth Edition) in Chapter 2: Clustering->Multicast Messaging for more information.

On the machine running NSA (SENDER) execute the following:

For UNIX:


export CLASSPATH=$NIKU_HOME/lib/jgroups-all.jar
 

For Windows:

set CLASSPATH=%NIKU_HOME%/lib/jgroups-all.jar

Then execute:


java org.jgroups.tests.McastSenderTest1_4 -mcast_addr <multicast_address> -port <beacon port> -use_all_interfaces
 

e.g. "java org.jgroups.tests.McastSenderTest1_4 -mcast_addr 239.255.252.3 -port 9090 -use_all_interfaces"

 

On the application servers (RECEIVER):

For UNIX:


export CLASSPATH=$NIKU_HOME/lib/jgroups-all.jar
 

For Windows:


set CLASSPATH=%NIKU_HOME%/lib/jgroups-all.jar
 

Then execute:

java org.jgroups.tests.McastReceiverTest1_4 -mcast_addr <multicast_address> -port <beacon> port -use_all_interfaces

e.g. java org.jgroups.tests.McastReceiverTest1_4 -mcast_addr 239.255.252.3 -port 9090 -use_all_interfaces

The same multicast address must be used on both the SENDER and on the RECEIVER. You should use the same multicast address/port that you specify in the NSA. If this fails you can try a different address/port and see if that succeeds.

Once the sender and receiver are running, you should be able to type a message in the terminal window of the "SENDER" - the same message should IMMEDIATELY appear in the window of the RECEIVER. This test should be done on each application server to see if it can RECEIVE messages from the SENDER.
The best way to test is to type in:

1 <enter>
2 <enter>
3 <enter>
4 <enter>
5 <enter>
...
30 <enter>

You should see every character you type in the SENDER replicated in the RECEIVER. If you don't, then your environment isn't passing multicast traffic. This could be because of a lack of IGMP Snooping being enabled on the switch in between the app servers.