Question

Consuming data from kafka does not include key

Forum|Forum|5 years ago
November 20, 2020
2 replies
5 views

jellang

I am able to consume messages from a kafka topic and store them in a flex table using a COPY from KafkaSource statement. This however does not get the key set by kafka for each message.
The messages have a simple key i.e.: i-01234567 and the message itself is a simple json with one value: { "loginAt": "2020-08-28T23:59:59Z"}
Is there a way to also get the key?

SergeB
Participating Frequently
Forum|Forum|5 years ago
November 25, 2020

Unfortunately, there is currently no mechanism to retrieve the Kafka message keys unless they're part of the message payload e.g { {"kafka_key":"i-01234567 ","loginAt": "2020-08-28T23:59:59Z"}