Skip to main content
Question

Could n't able to restart vertica database

  • Jun 14, 2019 6:47 AM UTC
  • 23 replies
  • 45 views

sprusty
Forum|alt.badge.img

Facing some issues during restarting vertica database by the following command:
admintools -t restart_node -F -s -d

The error was saying "Node was not down. Can't restart"

23 replies

Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • Jun 14, 2019 1:28 PM UTC

Are you trying to restart the entire DB or just a node in the DB?


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • Jun 14, 2019 1:36 PM UTC

I am trying to restart the entire vertica database


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • Jun 14, 2019 2:36 PM UTC

Then try using the start_db admintool.

admintools -t start_db -d <<dbname>>


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • Jun 14, 2019 2:41 PM UTC

It was giving me the below issue "Error: Shutdown: aborting shutdown"


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • Jun 14, 2019 2:46 PM UTC

What do you get from this command?

admintools -t list_allnodes

sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • Jun 14, 2019 2:49 PM UTC

Node | Host | State | Version | DB
-----------------+---------------+-------+-------------------+------
v_pmdb_node0001 | 172.27.21.249 | UP | vertica-7.1.2.000 | pmdb


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • Jun 14, 2019 2:50 PM UTC

Do you have a single node cluster? If so, looks like it is up. Can you connect to the pmdb database using vsql?


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • Jun 14, 2019 2:51 PM UTC

Yes We have a single node cluster. We can also connect using vsql


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • Jun 14, 2019 2:54 PM UTC

I'm confused. So you want to restart the database that is up? Why?

Anyway, if that's what you want to do, do this:

admintools -t stop_db -d pmdb
admintools -t start_db -d pmdb

sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • Jun 14, 2019 2:59 PM UTC

I have already tried to stop and then start it. Still facing the above issue which saying Error: Shutdown.

We are trying to restart it as there was an issue with the report extraction in OBR.
The issue is as below:
"The following database error occured: [Vertica]VJDBC ERROR FileColumnReader block /vertica/datafiles/pmdb/v_pmdb_node0001_data/129/45035999921377129/45035999921377129_0.fdb @ 0's CRC 35ae46f1f9e0c65a doesn't match record 2203c0ee4b544bc2."


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • Jun 14, 2019 3:16 PM UTC

sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • Jun 15, 2019 7:33 AM UTC

We are not able to start the vertica databse now with the above command. Can you please help.


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • Jun 18, 2019 8:07 AM UTC

Even after restoring the vertica database, still not able to start the database. Can you please help ?


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • Jun 18, 2019 9:41 AM UTC

What does this command return?

admintools -t return_epoch -d pmdb


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • Jun 18, 2019 11:09 AM UTC

LastEpoch|0


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • Jun 18, 2019 12:10 PM UTC

Hi Jim, Can we discuss over a call if you have time ?


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • Jun 18, 2019 4:16 PM UTC

Hi Jim, We have used the below command to start the vertica db.
admintools -t start_db -d pmdb -F

However It's not getting completely started ie in the list 2 status is INITIALIZING and rest of the node status is DOWN.

Can you please help.


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • Jun 19, 2019 6:52 PM UTC


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • Jun 20, 2019 3:02 AM UTC

@sprusty - Did you get the node started?

If not, try the attached script to manually start the node. I had to name the file with the .txt extension to upload it. Rename it to manual_start_up.sh when you DL it.

Copy the file to your node and make sure it is executable. Then run it like this:

./manual_start_up.sh pmdb


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • Jun 20, 2019 3:36 AM UTC

I'd also try renaming the corrupt file to something else:

mv /vertica/datafiles/pmdb/v_pmdb_node0001_data/129/45035999921377129/45035999921377129_0.fdb /vertica/datafiles/pmdb/v_pmdb_node0001_data/129/45035999921377129/45035999921377129_0.fdb_old

Then try to start the DB:

admintools -t start_db -d pmdb -F

Normaly Vertica will recover the file, but since you only have one node that's not going to happen. So you might lose some data.


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • Jun 20, 2019 5:23 AM UTC

Hi Jim, During running the script it was giving me an error saying "/bin/bash^M: bad interpreter: No such file or directory". So I ran a command sed -i -e 's/\r$//' scriptname.sh and then executed the same. During executing the script it said, database is up, however when checking it was showing DOWN.

INFO: Database pmdb is not running

Kindly help


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • Jun 20, 2019 12:43 PM UTC

@sprusty - Did you try renaming that file?


sprusty
Forum|alt.badge.img
  • Author
  • Participating Frequently
  • Jun 20, 2019 12:47 PM UTC

Thanks Jim for your continuous response on this.
Currently the DB is up after setting the projections to 0.

What should be our next course of action.
Regards
Suryakanta