eHealth will not start or continually crashes, what do I do?

Document ID : KB000032042
Last Modified Date : 14/02/2018
Show Technical Document Details

Problem/Symptoms: 

Common steps to take and consider when you are unable to start eHealth

Environment:  

All supported eHealth versions and their associated Operating Systems

Causes/Resolutions/Workarounds: 

Do a full recycle of eHealth services:
1. Stop eHealth.
     a. nhServer stop
2. Stop the database.
     a. nhStopDb immediate
3. Stop the web.
     a. Unix/Linux – nhHttpd stop
     b. Windows – Stop the following services in the services control panel (services.msc)
           i. eHealth httpd6x
           ii. ehealth Tomcat6x
4. Check for hung processes.  If eHealth and Oracle were shut down properly, the only things that should display should be the queries.
     a. ps –aef | grep –I nhi
     b. ps –aef | grep –I ora
           i. If any processes are still hung, please run the following command and attach its output to a new case.
                1. nhGetSupportInfo –type configuration 
5. Restart the web.
     a. Unix/Linux – nhHttpd start
     b. Windows – Start the following services in the services control panel (services.msc)
           i. eHealth httpd6x
           ii. ehealth Tomcat6x
6. Start the database.
     a. nhDbStart
7. Start eHealth.
     a. nhServer start

If the system still does not start, check the following logs for any indication as to why:
1. eHealth system log
     a. $NH_HOME/log/system.log
2. eHealth server_process log
     a. $NH_HOME/log/server_process.log
3. Oracle alert log
     a. Oracle 10g
            i. $ORACLE_HOME/admin/bdump/$ORACLE_SID/alert_$ORACLE_SID.log
     b. Oracle 11g
           i. $ORACLE_HOME/diagnostics/diag/rdbms/$ORACLE_SID(in lowercase)/$ORACLE_SID/trace/alert_$ORACLE_SID.log
4. Windows Event Log

Some examples of errors and situations to watch for:
1. Out of drivespace.  Ensure $NH_HOME, $ORACLE_HOME and the directory for the database are not out of / low on space.
2. Make sure Oracle is up and online before trying to start eHealth.
     a. ehealth]$ nhServer start
        Error: nhiCheckOracle requires ORACLE to be running and accessible,
        but it doesn't seem to be. Please start ORACLE and try again

3. $NH_HOME/log/system.log shows errors like the following:
     a. nhiPoller[Net] Pgm nhiPoller[Net]: Error reading (possibly missing) MTF '/opt/ehealth63/poller/mib2-enet-fd-nu-v2c-hs-IF-640.mtf', corresponding elements cannot be polled. 
     b. Solution: TEC614941
     http://www.ca.com/us/support/ca-support-online/product-content/knowledgebase-articles/tec614941.aspx

4. Orphaned Elements, Parents or MTF files.
     a. The following 3 nhisql commands should output 0 but if they do output data, forward it over to support in a new case via support.ca.com.
           i. $NH_HOME/bin/sys/nhisql “select element_id from nh_elem_assoc where element_id not in(select element_id from nh_element)”
           ii. $NH_HOME/bin/sys/nhisql “select parent_id from nh_elem_assoc where parent_id not in(select element_id from nh_element)”
           iii. $NH_HOME/bin/sys/nhisql “SELECT element_id, name, mtf_name FROM nh_element WHERE mtf_name not like ('%.mtf')”

5. AntiVirus software could have the needed files under lock for scanning.  Especially if they are Oracle / Database files this can lead to corruption and loss of data.  You should exclude $NH_HOME, $ORACLE_HOME, the install directory of the Database and the save directory of backup from being scanned to prevent this.

6. eHealth has run out of memory.  You will see errors in the $NH_HOME/log/system.log such as:

     a. Internal Error nhiDbServer Pgm nhiDbServer: Unexpected Null value for 'C++ constructor' (possibly out of memory). (ccm/newFailed)
     b. This is likely due to either the system being significantly under spec or you are exceeding the element limit for the operating system you are using.  After exceeding the count the system has the probability to become unstable and likely crash.  The further you exceed the count the higher the chances are.
           i. Windows – 50,000
           ii. Linux – 75,000
           iii. Solaris – 100,000

 

Additional Information:

If at any point you need to open a case, to help support get a head start on analysis, open a new case at support.ca.com and attach the output zip of the following command to the new case:


nhGetSupportInfo –type configuration