I’ve got a small, local client that I help out from time to time, and they recently had an issue where a node went down (3 node cluster), and their ETL and their connectivity didn’t work right. Near as I can tell, the Vertica cluster remained up, but while their down node was recovering, their load balancer decided it was OK to route queries there, even though the Vertica host wasn’t ready to receive requests, and it caused a lot of problems. So, the client and I definitely feel like it’s an issue with how their load balancer is set up.
I asked what they’re using, and he mentioned that it’s a simple Linux program called balance. I can’t help but wonder if maybe that’s inadequate for what we need, and maybe they should consider something like IPVS. They're also doing silly things like directly connecting to hostname/IP addresses, which doesn't sound like a terribly solid idea. Do we have any good recommendations around this space so they can shore this up? What are you seeing clients use?
