App service cannot be reached, Could not retrieve a valid tenant message thrown in logs

Document ID : KB000038923
Last Modified Date : 14/02/2018
Show Technical Document Details

Issue:

This issue occurs on a customer Oracle RAC issue, the loginTimeout in TenantConfigurationManager.java is dropping all the connections when they take beyond 5 sec. On some environments this is causing an issue : errors are thrown in the logs, the database servers are unavailable, systems are in outage.

Error messages that can be seen in the logs:
ERROR 2016-02-22 02:25:01,376 [http-nio-7020-exec-149] filter.RequestTenantIdExtractor (none:none:none:none)
com.niku.union.config.ConfigurationException: Could not retrieve a valid tenant.
at com.niku.union.config.TenantConfigurationManager.getValidLocalTenant(TenantConfigurationManager.java:355)
at com.niku.union.web.filter.RequestTenantIdExtractor.getTenantIdForRequest(RequestTenantIdExtractor.java:228)
at com.niku.union.web.filter.TenantSetupFilter.doFilter(TenantSetupFilter.java:49)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at com.niku.union.web.filter.CharsetFilter.doFilter(CharsetFilter.java:45) 

We also note random restarting of the affected services for app and bg.

Steps to Reproduce:

  1. On a system with a slower connections to the database, install CA PPM 14.3
  2. Note that the database connections do not get created when they hit 5 sec

Expected Result: App service cannot be reached, tenant is marked as invalid

Actual Result: App service should be reachable, tenant is still valid   

Cause:

Caused by CLRT-79927 

When troubleshooting with Oracle Support they indicated this is a timeout set on the client. 

Resolution: 

Resolved in CA PPM 14.3 Generic Patch #5   

Additional Information:

Reference TEC1696455 : PPM 14.3 Generic Patch available to resolve reported issues