Skip to main content

KafkaAvroParser and Schema Registry

  • July 12, 2017
  • 0 replies
  • 4 views

Francois
Forum|alt.badge.img+1

Following customer question, I've done some test of the KafkaAvroParser with schema registry

I use the following link to set up a test env : http://docs.confluent.io/current/quickstart.html
I've got this working (using flex table)
COPY test_avro_tgt source kafkasource(stream='test|0|-2', brokers='localhost:9092', stop_on_eof=true, duration=interval '10 seconds')
parser KafkaAvroParser(schema_registry_url='localhost:8081/');

But this one does not work (load three rows but with Null)
COPY test_avro_tgt
SOURCE KafkaSource(stream='test|0|-2', brokers='localhost:9092', stop_on_eof=true)
PARSER KafkaAVROParser(external_schema = '{"type":"record","name":"myrecord","fields":[{"name":"f1","type":"string"}]}')

So I look for a clarification about the documentation where we have : "Specifies the URL of the Confluent schema registry. This parameter is required to load data based on a schema registry version. If you are using an external schema, do not use this parameter. For more information, refer to Using a Schema Registry with Kafka."

  • Does it mean that we can or not use a syntax with the external_schema parameter like this one in parallel to the use of schema registry ?
  • If we are supposed to be able to use both syntax, where is my mistake ?

Regards