Skip to main content
Question

Is there any way to promote secondary subcluster to primary subcluster?

  • August 18, 2022
  • 3 replies
  • 11 views

cyun

Is there any way to promote secondary subcluster to primary subcluster when primary subcluster was down?
If there are two or more primary subclusters with the same number of nodes and specifications, I wonder if there is a way to use DML without restrictions even if a specific primary subcluster goes down.

dbadmin=> select version();
               version               
-------------------------------------
 Vertica Analytic Database v12.0.0-0

-- Even if I have two primary subclusters, if one primary subcluster goes down, the remaining clusters(including primary subcluster) are converted to read-only mode.

dbadmin=> select subcluster_name, node_name, node_state, is_primary, is_readonly from nodes order by node_name;
 subcluster_name |     node_name     | node_state | is_primary | is_readonly 
-----------------+-------------------+------------+------------+-------------
 sc1             | v_vertdb_node0001 | UP         | t          | f
 sc1             | v_vertdb_node0002 | UP         | t          | f
 sc1             | v_vertdb_node0003 | UP         | t          | f
 sc2             | v_vertdb_node0004 | UP         | t          | f
 sc2             | v_vertdb_node0005 | UP         | t          | f
 sc2             | v_vertdb_node0006 | UP         | t          | f
 sc3             | v_vertdb_node0007 | UP         | f          | f
 sc3             | v_vertdb_node0008 | UP         | f          | f
 sc3             | v_vertdb_node0009 | UP         | f          | f

-- primary subcluster #2 shutdown

dbadmin=> select shutdown_subcluster('sc2');

dbadmin=> select subcluster_name, node_name, node_state, is_primary, is_readonly from nodes order by node_name;
 subcluster_name |     node_name     | node_state | is_primary | is_readonly 
-----------------+-------------------+------------+------------+-------------
 sc1             | v_vertdb_node0001 | UP         | t          | t
 sc1             | v_vertdb_node0002 | UP         | t          | t
 sc1             | v_vertdb_node0003 | UP         | t          | t
 sc2             | v_vertdb_node0004 | DOWN       | t          | t
 sc2             | v_vertdb_node0005 | DOWN       | t          | t
 sc2             | v_vertdb_node0006 | DOWN       | t          | t
 sc3             | v_vertdb_node0007 | UP         | f          | t
 sc3             | v_vertdb_node0008 | UP         | f          | t
 sc3             | v_vertdb_node0009 | UP         | f          | t
(9 rows)


dbadmin=> create table t1(c1 int);
ERROR 10428:  Transaction commit aborted since the database is currently in read-only mode
HINT:  Commits will be restored when the database restores the quorum

-- If I have an additional secondary subcluster in one primary subcluster, the secondary subcluster cannot be promoted to the primary subcluster after the primary subcluster is down.

dbadmin=> select demote_subcluster_to_secondary('sc2');
 demote_subcluster_to_secondary 
--------------------------------
 DEMOTE SUBCLUSTER TO SECONDARY
(1 row)

dbadmin=> 
dbadmin=> select subcluster_name, node_name, node_state, is_primary, is_readonly from nodes order by node_name;
 subcluster_name |     node_name     | node_state | is_primary | is_readonly 
-----------------+-------------------+------------+------------+-------------
 sc1             | v_vertdb_node0001 | UP         | t          | f
 sc1             | v_vertdb_node0002 | UP         | t          | f
 sc1             | v_vertdb_node0003 | UP         | t          | f
 sc2             | v_vertdb_node0004 | UP         | f          | f
 sc2             | v_vertdb_node0005 | UP         | f          | f
 sc2             | v_vertdb_node0006 | UP         | f          | f
 sc3             | v_vertdb_node0007 | UP         | f          | f
 sc3             | v_vertdb_node0008 | UP         | f          | f
 sc3             | v_vertdb_node0009 | UP         | f          | f
(9 rows)


dbadmin=> select shutdown_subcluster('sc1');

 shutdown_subcluster 
---------------------
 Subcluster shutdown


dbadmin=> select subcluster_name, node_name, node_state, is_primary, is_readonly from nodes order by node_name;
 subcluster_name |     node_name     | node_state | is_primary | is_readonly 
-----------------+-------------------+------------+------------+-------------
 sc1             | v_vertdb_node0001 | DOWN       | t          | t
 sc1             | v_vertdb_node0002 | DOWN       | t          | t
 sc1             | v_vertdb_node0003 | DOWN       | t          | t
 sc2             | v_vertdb_node0004 | UP         | f          | t
 sc2             | v_vertdb_node0005 | UP         | f          | t
 sc2             | v_vertdb_node0006 | UP         | f          | t
 sc3             | v_vertdb_node0007 | UP         | f          | t
 sc3             | v_vertdb_node0008 | UP         | f          | t
 sc3             | v_vertdb_node0009 | UP         | f          | t
(9 rows)


dbadmin=> select promote_subcluster_to_primary('sc2');
ERROR 10551:  Modifying subclusters is not supported in read-only mode
HINT:  The operation will be supported when the database is restored to full mode

3 replies

SruthiA
Forum|alt.badge.img+1
  • Participating Frequently
  • August 25, 2022

All the primary nodes work together to maintain the data in the shards. even if one primary subcluster is down, cluster will go into read only mode. In read only, you cannot promote secondary subcluster to primary. it is expected behavior. for more information, please visit the below link

https://www.vertica.com/docs/12.0.x/HTML/Content/Authoring/Eon/MaintainingDataIntegrityInEonMode.htm


cyun
  • Author
  • New Participant
  • September 7, 2022

Thank you for your reply.
I have an additional question.
If there are 2 primary subclusters, can you explain in more detail why if one primary subcluster goes down, the remaining primary subcluster cannot maintain quorum?


mosheg
Forum|alt.badge.img+2
  • Participating Frequently
  • September 7, 2022

There are two requirements for the database to continue normal operations: maintaining quorum, and maintaining shard coverage.
SELECT * FROM critical_subclusters; will list the EON primary subclusters whose loss would cause the database to become unsafe and force it to shutdown.
See: https://www.vertica.com/docs/12.0.x/HTML/Content/Authoring/Eon/MaintainingDataIntegrityInEonMode.htm