Multi Port Collector MTP is hanging in state of initializing

Document ID : KB000020381
Last Modified Date : 14/02/2018
Show Technical Document Details

Description:

Symptoms;

Your MTP collector has stopped.
And maybe the system is down since a power outage? And or a network interruption?

The Analysis page on the collector is hanging (update circle running) and showing error;
"Could not open database connection. Error in SqlCommand at Execute:"

On the SA Admin page the collector status is stopped and it cannot be started (action failed).

All attempts to start the DB are futile and after a reboot the DB status goes into state of "INITIALIZING"

Did you have a power outage? It's possible that the Vertica database has been corrupted.
And it's possible you may need to stop and recreate the database.

If in any doubt about following these steps, then please raise a support case with CA support.

Solution:

Fix steps, if stop and recreate is required.

  1. Check name resolution. Edit /etc/hosts

  2. Stop the processes that directly access the Vertica database.
    sudo /sbin/service nqwatchdog stop
    sudo /sbin/service nqinspectoragentd stop

  3. Drop database:
    [root@mtp ~]# /etc/init.d/verticad stop
    [root@mtp ~]# su - dbadmin -c "/opt/vertica/bin/adminTools -t drop_db -d
    capture"

  4. Confirm that there is no Vertica database process running

  5. Restart the Vertica spreadd daemon.
    sudo /sbin/service spreadd stop
    sudo /sbin/service spreadd start

  6. Recreate the database.
    sudo /opt/NetQoS/install/setupVertica.sh --new

  7. Restart nqinspectoragentd and nqwatchdog
    sudo /sbin/service nqinspectoragentd start
    sudo /sbin/service nqwatchdog start

All should now be fine.