Seeing caught exception when determining the difference between MOM and a Collector's harvest time: Collector xx.xx.xx.xx@5001: com.wily.isengard.message.MessageUndeliverableException: Outgoing mailbox is closed. Message cannot be sent

Document ID : KB000044107
Last Modified Date : 14/02/2018
Show Technical Document Details

Question: 

 I keep seeing the below errors on my Collector.  What do they mean?

[ERROR] [Collector xx.xx.xx.xx@5001] [Manager.Cluster] Caught exception trying to get the difference between MOM and this Collector's harvest time: Collector xx.xx.xx.xx@5001: com.wily.isengard.message.MessageUndeliverableException: Outgoing mailbox is closed. Message cannot be sent

 

[ERROR] [pool-11-thread-1] [Manager] Uncaught Exception in Enterprise Manager:  In thread pool-11-thread-1 and the message is com.wily.util.exception.UnexpectedExceptionError: Tranport for the registry service at address: {1} is down

 

 Environment:  

 All APM versions

 

 Answer: 

  These messages mean that the number of time slices index for the Collectors harvest is either behind or ahead of the MOM's harvest.  In either case, the Collector will disconnect from the MOM.

 

 When these messages are seen, the Collector requests a disconnect. 

 

 If this error is happening, then check the following:

 1. Are all the transport settings set on the MOM and Collectors?

     transport.outgoingMessageQueueSize

     transport.override.isengard.high.concurrency.pool.min.size

     transport.override.isengard.high.concurrency.pool.max.size

 

 2. Are the EM's on a virtual server?  Sometimes not enough resources by the host server are being distributed out to all VM's equally.

 

 Anytime you change any one of the above transport properties, a restart is required.  After doing this change and restart, if you still experience these messages, then examine your VM's (if the EM resides on a VM) and check your VM hosting server to ensure proper resources are being given to them and that nothing is being choked.

 

After this if needing further assistance from APM Support, then we highly recommend opening an issue.  In the issue, upload the entire logs directory and config directory from MOM and all Collectors so that we can analyze your environment and see if anything else is affecting the performance.