Skip to main content
Question

Could n't able to restart vertica database

  • June 14, 2019
  • 23 replies
  • 41 views

sprusty
Forum|alt.badge.img

Facing some issues during restarting vertica database by the following command:
admintools -t restart_node -F -s -d

The error was saying "Node was not down. Can't restart"

23 replies

Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • June 14, 2019

Are you trying to restart the entire DB or just a node in the DB?


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • June 14, 2019

I am trying to restart the entire vertica database


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • June 14, 2019

Then try using the start_db admintool.

admintools -t start_db -d <<dbname>>


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • June 14, 2019

It was giving me the below issue "Error: Shutdown: aborting shutdown"


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • June 14, 2019

What do you get from this command?

admintools -t list_allnodes

sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • June 14, 2019

Node | Host | State | Version | DB
-----------------+---------------+-------+-------------------+------
v_pmdb_node0001 | 172.27.21.249 | UP | vertica-7.1.2.000 | pmdb


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • June 14, 2019

Do you have a single node cluster? If so, looks like it is up. Can you connect to the pmdb database using vsql?


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • June 14, 2019

Yes We have a single node cluster. We can also connect using vsql


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • June 14, 2019

I'm confused. So you want to restart the database that is up? Why?

Anyway, if that's what you want to do, do this:

admintools -t stop_db -d pmdb
admintools -t start_db -d pmdb

sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • June 14, 2019

I have already tried to stop and then start it. Still facing the above issue which saying Error: Shutdown.

We are trying to restart it as there was an issue with the report extraction in OBR.
The issue is as below:
"The following database error occured: [Vertica]VJDBC ERROR FileColumnReader block /vertica/datafiles/pmdb/v_pmdb_node0001_data/129/45035999921377129/45035999921377129_0.fdb @ 0's CRC 35ae46f1f9e0c65a doesn't match record 2203c0ee4b544bc2."


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • June 14, 2019

sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • June 15, 2019

We are not able to start the vertica databse now with the above command. Can you please help.


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • June 18, 2019

Even after restoring the vertica database, still not able to start the database. Can you please help ?


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • June 18, 2019

What does this command return?

admintools -t return_epoch -d pmdb


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • June 18, 2019

LastEpoch|0


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • June 18, 2019

Hi Jim, Can we discuss over a call if you have time ?


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • June 18, 2019

Hi Jim, We have used the below command to start the vertica db.
admintools -t start_db -d pmdb -F

However It's not getting completely started ie in the list 2 status is INITIALIZING and rest of the node status is DOWN.

Can you please help.


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • June 19, 2019


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • June 20, 2019

@sprusty - Did you get the node started?

If not, try the attached script to manually start the node. I had to name the file with the .txt extension to upload it. Rename it to manual_start_up.sh when you DL it.

Copy the file to your node and make sure it is executable. Then run it like this:

./manual_start_up.sh pmdb


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • June 20, 2019

I'd also try renaming the corrupt file to something else:

mv /vertica/datafiles/pmdb/v_pmdb_node0001_data/129/45035999921377129/45035999921377129_0.fdb /vertica/datafiles/pmdb/v_pmdb_node0001_data/129/45035999921377129/45035999921377129_0.fdb_old

Then try to start the DB:

admintools -t start_db -d pmdb -F

Normaly Vertica will recover the file, but since you only have one node that's not going to happen. So you might lose some data.


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • June 20, 2019

Hi Jim, During running the script it was giving me an error saying "/bin/bash^M: bad interpreter: No such file or directory". So I ran a command sed -i -e 's/\r$//' scriptname.sh and then executed the same. During executing the script it said, database is up, however when checking it was showing DOWN.

INFO: Database pmdb is not running

Kindly help


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • June 20, 2019

@sprusty - Did you try renaming that file?


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • June 20, 2019

Thanks Jim for your continuous response on this.
Currently the DB is up after setting the projections to 0.

What should be our next course of action.
Regards
Suryakanta