AdminUI frozen and handshake errors due to Socket error 0

Document ID : KB000107619
Last Modified Date : 30/07/2018
Show Technical Document Details
Question:
We have installed AdminUI and once registered with XPSRegClient tool, when we go to access it through the browser for the first time, after entering superuser credentials and Policy Server hostname, the AdminUI seems to be frozen showing a wheel but never ending. AdminUI is installed in a different server than the Policy Server, but they are both in the same subnet and running in RedHat OS.

We have noticed the following errors in the smps.log file:

[2783/140546697852672][Mon Jul 23 2018 11:40:43][CServer.cpp:2121][ERROR][sm-Tunnel-00010] Bad security handshake attempt. Handshake error: 3152
[2783/140546697852672][Mon Jul 23 2018 11:40:43][CServer.cpp:2128][ERROR][sm-Tunnel-00030] Handshake error: Failed to receive client hello. Socket error 0
[2783/140546697852672][Mon Jul 23 2018 11:40:43][CServer.cpp:2293][ERROR][sm-Server-01070] Failed handshake with ::ffff:10.10.10.10:38428
[2783/140544953022208][Mon Jul 23 2018 11:40:49][PolicyCache.cpp:1307][INFO][sm-Server-02880] Building policy cache ...
[2783/140544953022208][Mon Jul 23 2018 11:40:49][PolicyCache.cpp:1406][INFO][sm-Server-02890] Building policy cache done
[2783/140546672674560][Mon Jul 23 2018 11:40:53][CServer.cpp:2121][ERROR][sm-Tunnel-00010] Bad security handshake attempt. Handshake error: 3152
[2783/140546672674560][Mon Jul 23 2018 11:40:53][CServer.cpp:2128][ERROR][sm-Tunnel-00030] Handshake error: Failed to receive client hello. Socket error 0
[2783/140546672674560][Mon Jul 23 2018 11:40:53][CServer.cpp:2293][ERROR][sm-Server-01070] Failed handshake with ::ffff:10.10.10.10:49034
[2783/140546177734400][Mon Jul 23 2018 11:41:10][CServer.cpp:1874][INFO][sm-Server-01770] Closing accepted connection for session  # 3 connection idle too long before handshake .
[2783/140546672674560][Mon Jul 23 2018 11:57:09][CServer.cpp:2121][ERROR][sm-Tunnel-00010] Bad security handshake attempt. Handshake error: 3156
[2783/140546672674560][Mon Jul 23 2018 11:57:09][CServer.cpp:2136][ERROR][sm-Tunnel-00070] Handshake error: Failed to receive client ack. Socket error 0
[2783/140546672674560][Mon Jul 23 2018 11:57:09][CServer.cpp:2293][ERROR][sm-Server-01070] Failed handshake with ::ffff:10.10.10.10:53402
[2783/140546177734400][Mon Jul 23 2018 11:57:55][CServer.cpp:1874][INFO][sm-Server-01770] Closing accepted connection for session  # 6 connection idle too long before handshake .
[2783/140546672674560][Mon Jul 23 2018 12:02:40][CServer.cpp:2121][ERROR][sm-Tunnel-00010] Bad security handshake attempt. Handshake error: 3152
[2783/140546672674560][Mon Jul 23 2018 12:02:40][CServer.cpp:2128][ERROR][sm-Tunnel-00030] Handshake error: Failed to receive client hello. Socket error 0
[2783/140546672674560][Mon Jul 23 2018 12:02:40][CServer.cpp:2293][ERROR][sm-Server-01070] Failed handshake with ::ffff:10.10.10.10:38448
[2783/140546681067264][Mon Jul 23 2018 12:50:34][CServer.cpp:2121][ERROR][sm-Tunnel-00010] Bad security handshake attempt. Handshake error: 3156
[2783/140546681067264][Mon Jul 23 2018 12:50:34][CServer.cpp:2136][ERROR][sm-Tunnel-00070] Handshake error: Failed to receive client ack. Socket error 0
[2783/140546681067264][Mon Jul 23 2018 12:50:34][CServer.cpp:2293][ERROR][sm-Server-01070] Failed handshake with ::ffff:10.10.10.10:53412
[2783/140546177734400][Mon Jul 23 2018 12:51:30][CServer.cpp:1874][INFO][sm-Server-01770] Closing accepted connection for session  # 9 connection idle too long before handshake .
[2783/140546177734400][Mon Jul 23 2018 12:54:00][CServer.cpp:1874][INFO][sm-Server-01770] Closing accepted connection for session  # 10 connection idle too long before handshake .
[2783/140546177734400][Mon Jul 23 2018 13:04:00][CServer.cpp:1874][INFO][sm-Server-01770] Closing accepted connection for session  # 11 connection idle too long before handshake .
[2783/140546177734400][Mon Jul 23 2018 13:06:20][CServer.cpp:1874][INFO][sm-Server-01770] Closing accepted connection for session  # 12 connection idle too long before handshake .
[2783/140546177734400][Mon Jul 23 2018 13:08:05][CServer.cpp:1874][INFO][sm-Server-01770] Closing accepted connection for session  # 13 connection idle too long before handshake .
[2783/140546177734400][Mon Jul 23 2018 13:16:40][CServer.cpp:1874][INFO][sm-Server-01770] Closing accepted connection for session  # 14 connection idle too long before handshake .
[2783/140546689459968][Mon Jul 23 2018 13:26:15][CServer.cpp:2121][ERROR][sm-Tunnel-00010] Bad security handshake attempt. Handshake error: 3156
[2783/140546689459968][Mon Jul 23 2018 13:26:15][CServer.cpp:2136][ERROR][sm-Tunnel-00070] Handshake error: Failed to receive client ack. Socket error 0
[2783/140546689459968][Mon Jul 23 2018 13:26:15][CServer.cpp:2293][ERROR][sm-Server-01070] Failed handshake with ::ffff:10.10.10.10:53442
[2783/140546681067264][Mon Jul 23 2018 13:29:32][CServer.cpp:2121][ERROR][sm-Tunnel-00010] Bad security handshake attempt. Handshake error: 3159
[2783/140546681067264][Mon Jul 23 2018 13:29:32][CServer.cpp:2126][ERROR][sm-Tunnel-00020] Handshake error: Failed to receive client hello. Client disconnected
[2783/140546681067264][Mon Jul 23 2018 13:29:32][CServer.cpp:2293][ERROR][sm-Server-01070] Failed handshake with ::ffff:10.10.10.10:53444


How can we solve this issue?
Environment:
AdminUI R12.8 in RHEL7
Policy Server R12.8 in RHEL7
Answer:
This kind of errors can happen when the entropy on the OS is too low, so you should first ensure that you have enough entropy on both servers. You can check the following documentation for entropy settings for both components: 
Policy Server : https://docops.ca.com/ca-single-sign-on/12-8/en/installing/install-a-policy-server/install-policy-server-on-unix/prepare-for-the-policy-server-installation#PrepareforthePolicyServerInstallation-IncreaseEntropy 
Also appearing in AdminUI : https://docops.ca.com/ca-single-sign-on/12-8/en/installing/install-the-administrative-ui/install-the-administrative-ui-on-linux-stand-alone 

You can run the following command to know how many entropy there is at that moment: 
cat /proc/sys/kernel/random/entropy_avail 
If it is too low, please, increase it following the information above.

Also, ensure the JCE patch is applied for the Java used by both the Policy Server and the AdminUI JBoss (or has enabled the unlimited cryptography settings as per the following KB: https://comm.support.ca.com/kb/how-to-apply-the-jce-patch-in-jdk18151-or-higher/kb000016726)