Hi,
I would like to know can Vertica vkconfig be set microbatch on Vertica node to dedicate specific Kafka partitions. During several test, I keep seeing that vkconfig reads source file when started and launch with all the partitions on every nodes.
What I would like to do here is that scheduler on a node to be distributed and dedicated to specific Kafka partition.
For example, if there are 3 node Vertica cluster and Kafka with 6 partition on a topic,
Kafka partition 0,1 ----> load to node 1 (dedicated)
Kafka partition 2,3 ----> load to node 2 (dedicated)
Kafka partition 4,5 ----> load to node 3 (dedicated)
It is same logic with doing so as below with COPY statement:
-- partition 0
COPY "public"."vendor_dimension" SOURCE KafkaSource(stream='test1|0|-2', brokers=...
Sign up
Already have an account? Login
Welcome to the Rocket Forum!
Please log in or register:
Employee Login | Registration Member Login | RegistrationEnter your E-mail address. We'll send you an e-mail with instructions to reset your password.