Skip to main content
Question

local segmentation

  • December 17, 2018
  • 5 replies
  • 19 views

An_kir
Forum|alt.badge.img+2
  • Participating Frequently

Hi AV,
a client is installing Vertica on 20 nodes and will expand to 30 soon. what are the recommendations for the local segmentation?

thanks!

5 replies

Yassine
  • New Participant
  • December 17, 2018
A first magic number for local segments would be 3. This would allow you to scale from 20 to 30 to 60 without skew nor re-segmentation.
The other option is 6 for 20 -> 30 -> 40 -> 60 -> 120.

Beware of 'too many ROS containers' though.

On 12/17/18, 9:02 AM, "An_kir" wrote:

Vertica Forum https://forum.vertica.com/
An_kir started a new discussion: local segmentation

Hi AV,

a client is installing Vertica on 20 nodes and will expand to 30 soon. what are the recommendations for the local segmentation?

thanks!

Reply to this email directly or follow the link below to check it out: https://forum.vertica.com/discussion/240226/local-segmentation

Car1os
Forum|alt.badge.img
  • Participating Frequently
  • December 17, 2018

If they are planning to expand very soon, then enabling local segmentation may be a good idea, but if this is not happening soon, then don't enable it. I don't believe we are recommending using this anymore.


Vertica_Curtis
Forum|alt.badge.img+1
  • Participating Frequently
  • December 17, 2018

We need to qualify "very soon". Local segmentation can be nice when expanding the size of a cluster, however it's not at all required. And while I agree that a local segmentation value of 3 might be ideal in this situation, that's going to multiply all their existing projections by 3, which has a good chance of creating ROS pushback errors.

The other thing to consider here is that you can't force a rebalance just due to the implementation of local segmentation. There's no way to actually subdivide the projections into local segments just because you enable the feature. It doesn't initiate any kind of rebalance, and the system will just have to take it's own time to create those local segments. So, if you plan on upgrading in a couple of days, turning on local segmentation will have no benefit whatsoever. If you plan on upgrading in 2-3 months, then it can have some benefit. You just have to handle the risks.


An_kir
Forum|alt.badge.img+2
  • Author
  • Participating Frequently
  • December 17, 2018

as we understand, the current plan is to use 20 nodes for about 1 Pb of data and to add 10 more nodes in about a year for another 0.5 Pb. if we do not use the local segmentation, is re-segmentation of a 100 Tb a real option?!


An_kir
Forum|alt.badge.img+2
  • Author
  • Participating Frequently
  • December 17, 2018

100 Tb is the estimation of a fact table size. they will have several of them. they will be partitioned somehow by date, if it is important.