The sysadmins for our database cluster would like to cycle through all nodes and, for each node, wipe/restructure the primary data disk partition. The basic strategy we'd planned to use was:
1) Remove Node 1 from cluster (rebalancing to ensure remaining nodes are still in K-SAFE1 mode during maintenance & recovery of node 1)
2) Perform maintenance on node 1 (wiping out all data)
3) Add Node 1 as new node
4) Wait for Node 1 to switch from RECOVERING to UP
5) Repeat 1 - 4 for all other nodes (2, 3, 4...)
However, according to the documentation, a prerequisite of removing a node is:
- The node must be empty, in other words there should be no projections referring to the node. Ensure you have followed the steps listed in Removing Nodes to modify your database design.
This leads me to think the strategy may be biting off way more work than necesary (i.e. would require a lot of projection-level operations before removing each node "safely").
Would it make more sense to:
1) Simply stop the database on node 1 (putting KSafe-1 cluster into critical state).
2) Perform maintenance on that node - wiping all data contents of the node.
3) Restart database on node 1.
4) Wait on Node 1 to recover
5) Repeat steps 1-4 for remaining nodes
Beyond operating the cluster in critical mode during the maintenance & recovery windows, are there any other unforseen hazards to this approach? Will Vertica recover each node cleanly, despite the fact that all pre-existing data will have been wiped out on that node?
