Not able to compute keys vertica kafka

Hi,

After a quick look by one of the R&D team members we think you may be using the base Kafka Parser and not the Kafka JSON Parser.

Check out this section of the documentation:

https://my.vertica.com/docs/7.2.x/HTML/index.htm#Authoring/KafkaIntegrationGuide/UsingCOPYwithKafka.htm%3FTocPath%3DIntegrating%2520with%2520Apache%2520Kafka%7C_____5

Thanks,

Mark

Hi Mark,

Thanks for your response.

With COPY statment, vertica is parsing the json input correctly. How to do it with scheduler?

I am using the following command to assign the json parser to scheduler.

/opt/vertica/packages/kafka/bin/vkconfig topic --add --target data_tgt --rejection-table data_rej --topic data --num-partitions 1 --config-schema data  --parser KafkaJSONParser

Hi phmandawa,

I am having exactly same problem as yours.

My vkconfig topic command is same as yours, but it seems it does not parsing json formatted message to the vertica flex table.

Another thing that I noticed is everytime I edit topic with --parser KafkaJSONParser option, it does not reflect in kafka_targets table. Example below shows KafkaParser all the time. Shouldn't it be changed to KafkaJSONParser?  I even tried to update column value manually to KafkaJSONParser, Kafka does not want to parse as JSON format.

target_schema | target_table | enabled | parser | rejection_table | parser_parameters | load_method | target_columns
---------------+--------------+---------+-------------+---------------------+-------------------+-------------+----------------
flextest1 | kafka | t | KafkaParser | flextest1.kafka_rej | | TRICKLE |

Have you solved this issue?  Please help if you have solved yours.

Thanks,

HI,

I could not find a solution. We went with a different approach.