unable to start mysql on primary node

Document ID : KB000098068
Last Modified Date : 14/09/2018
Show Technical Document Details
Question:
The user is  getting below error while starting and stopping the mysql on primary node. [root@XXX01 mysql]# service mysql restart ERROR! MySQL server PID file could not be found! Starting MySQL...
ERROR! The server quit without updating PID file (/var/lib/mysql/mysqld.pid). [root@XXX01 mysql]# service mysql stop ERROR! MySQL server PID file could not be found! [root@XXX01 mysql]# service mysql start Starting MySQL... ERROR!
The server quit without updating PID file (/var/lib/mysql/mysqld.pid). [root@XXX01 mysql]#
Environment:
ENV 9.3 CR01
Answer:
We have seen this error when the gateway appliance or the server running MySQL is shutdown incorrectly causing the log file to become corrupt or the one we see more often is the hard drive fills up and bin-log files are removed causing the MySQL server to be unable to restart.

The best approach is to

1) First of all, Backup the /var/lib/mysql directory 

2) Remove all the ssgbin log files and update the ssgbin-log.index file to reflect 000001 for the new bin-log file 

3) Start the MySQL service 

4) Stop slave on both the primary and secondary DB nodes "mysqladmin stop-slave" 

5) Rebuild the replication using the create slave script as descibed in the doc below:
    https://comm.support.ca.com/kb/reinitialize-replication-in-a-multinode-cluster/kb000044402