Skip to main content
Question

Eon: Force termination of compute nodes

  • December 5, 2018
  • 2 replies
  • 16 views

LenoyJ
Forum|alt.badge.img+1
  • Participating Frequently

How resilient is Eon when compute nodes are forcefully terminated (using the EC2 console) in an unclean fashion (without stopping the DB)? Can we still use revive_db with brand new compute nodes? If not, how do we get the DB back up?

Assuming that the communal storage was synced with the latest metadata. I'm trying to figure out what happens in a catastrophic failure of compute nodes.

2 replies

Car1os
Forum|alt.badge.img
  • Participating Frequently
  • December 5, 2018
Based on my understanding, you should be able to revive the DB, there is a lock that I think expire after 5 minutes, once it expires you can revive the DB.

Don't remember where I read about the lock, but if you terminate the instances and try to revive the MC will display a message about a lock.




On 12/4/18, 10:52 PM, "LenoyJ" wrote:

Vertica Forum https://forum.vertica.com/
LenoyJ started a new discussion: Eon: Force termination of compute nodes

How resilient is Eon when compute nodes are forcefully terminated (using the EC2 console) in an unclean fashion (without stopping the DB)? Can we still use revive_db with brand new compute nodes? If not, how do we get the DB back up?

Assuming that the communal storage was synced with the latest metadata. I'm trying to figure out what happens in a catastrophic failure of compute nodes.

Reply to this email directly or follow the link below to check it out: https://forum.vertica.com/discussion/240181/eon-force-termination-of-compute-nodes

LenoyJ
Forum|alt.badge.img+1
  • Author
  • Participating Frequently
  • December 6, 2018

Tested this out. Yes, revive_db works even when compute nodes are terminated without a clean shutdown! And it did say, "Cluster lease has expired, it is safe to revive." Though there is no documentation on how long this lock is held. (I ran revive_db well after 5 minutes though).