Implications of k-safety = 0?
What are the performance implications of running a four-node cluster with k-safety set to zero rather than one? We run a cluster that is reloaded on a regular basis and often includes transient tables. It would appear that we could save a significant amount of storage if we were to reduce k-safety from zero to 1 and eliminate the buddy projections. I understand the downsides as far as the cluster shutting down on a single node failure (until the entire cluster could be reloaded). If that's acceptable, are there other positive or negative issues to consider in reducing k safety to zero?
+2Hi Steve, thanks for your question. We'll check with the team to see what implications this action may have.
+2Well, it means that you really don't care about your data. It's like running a disk array at RAID-0 -- sure, you get more disk space; but if you lose a drive, even temporarily, odds are that everything gets thrown out and you have to reformat the array (or, in this case, the cluster) and wipe out all your data. My experience has been that most users are totally fine with this right up until the point where a node actually crashes... Having fewer projections (reducing K-safety just reduces the minimum number of projections per table) can actually increase load performance -- fewer copies of the data to sort and write. But it means you have fewer chances to optimize your data for your query workload.
Sign up
Already have an account? Login
Welcome to the Rocket Forum!
Please log in or register:
Employee Login | Registration Member Login | RegistrationEnter your E-mail address. We'll send you an e-mail with instructions to reset your password.
77 4th Avenue
Waltham, MA 02451 USA
