We have performed the installation of AdminUI R12.7, we registered it with our Policy Server, it starts correctly and after some time it begins to fail returning HTTP 404 errors. Afterwards, it does not restart correctly anymore unless it is reinstalled.
How can we solve this issue?
This problem was caused as the open files limit in the OS was reached and AdminUI failed to open some files needed:
2018-07-10 16:41:16,244 WARN [org.jboss.as.connector.subsystems.datasources.AbstractDataSourceService$WildFlyLocalMCF] (ConnectionValidator) Destroying connection that is not valid, due to the following exception: org.apache.derby.impl.jdbc.EmbedConnection40@613608241 (XID = null), (SESSIONID = 1), (DATABASE = /opt/siteminder/adminui/standalone/data/derby/siteminder/taskpersistence), (DRDAID = null) : java.sql.SQLException: Meta-data for Container org.apache.derby.impl.store.raw.data.RAFContainer4@346ea6d1 could not be accessed
Caused by: java.sql.SQLException: Java exception: '/opt/siteminder/adminui/standalone/data/derby/siteminder/taskpersistence/seg0/c2f0.dat (Too many open files): java.io.FileNotFoundException'.
Then other errors were followed and causing the HTTP 404 errors. During the startup of the AdminUI there is a warning letting you know that the current open file limit is set too low:
2018-07-10 16:07:04,356 WARN [org.jboss.as.warn.fd-limit] (main) JBAS015972: The operating system has limited the number of open files to 1024 for this process; a value of at least 4096 is recommended
Increasing the open file limit at OS level will solve this issue.