One machine from Cluster environment is going down

Document ID : KB000091706
Last Modified Date : 23/04/2018
Show Technical Document Details
Issue:
One of the machines that has Vertica DB went down
Only one machine is going down.0.4 ENV: DEV/TEST ENT: YES KB & COMMUNITIES: HAVE NOT FOUND ANYTHING RELEVANT
Environment:
Pm 3.2
VERTICA: 8.1.0.4
Cause:
k-safe security syncronization
 
Resolution:
If this a k-safe cluster and other nodes are UP, then please run below command on the down node. This command will start the 'node' in force mode, and it will remove all the corrupt data files and same will be recovered from other nodes in the cluster.

$admintools -t restart_node -F -s <this_Hostname_or_IP> -d <dbname>

Please replace hostname/IP and database name in the command.

Note1: This should to stop all nodes for synchronization and start using /opt/vertica/bin/adminTools so try to do this command out of business hours because it can take a long time to recover

Note2: Before executing this command run the Vertica Backup of node that is working fine
Additional Information:
For standalone environment see  Document ID : KB000037369
Unable to start a single node Vertica database
https://comm.support.ca.com/kb/unable-to-start-a-single-node-vertica-database/kb000037369