Vertica Crash

As I know there is no any standard procedure. But to investigate the issue it's good to provide log from all the nodes.

You can also use below command to capture the log from all the nodes and zip it.

Log in as the dbadmin user and execute  /opt/vertica/bin/diagnostics
   If you have multiple databases on your system, use the "-d <database name> argument when running the diagnostics.

   Diagnostics creates a file named /opt/vertica/log/VerticaDiagnostics.<date>.zip
   where date is replaced with a numeric date representation so that multiple calls to diagnostics, result in different files being created.
   The diagnostics utility collects the information from all nodes in your cluster. You only need to run it on one node in the cluster. 

Hi,

Also you can go through the Vertica logs on each node or the node which got crashed and review the logs. There are few keywords that you can look for which are : PANIC, ERRORS,WARNINGS and copy those few lines here to review.

As Nimmi said you can capture the whole logs from each node.



Regards

Bhawana

~path/$dbname/dblog  -- will give you information for information on crash if any recorded by the process dump

~path/$catalog_path/ErrorReport.txt -- same as above but some times the OS node crash itself does'nt cannot write to the logs dumping the process information.

~path/$catalog_path/vertica.log -- The logs gives you a clue for what was processing just before the crash. Every restart of the Database or Node writes the logs starting as New log

The First two might give you information which you cannot process or understand as the functions names are printed and used by us to determine the bug in the exact piece of code.

The above exact path and dbname can be found in the file /opt/vertica/config/admintools.conf

Once the database is up instead of looking at the vertica.log you can get the same information from the database via query too
select * from error_messages where error_level='ERROR';

Replace the variable values from the list below.
WARNING
 ERROR
 PANIC
 FATAL