Skip to main content

Data ingestion from MapR Streams to Vertica

  • June 29, 2017
  • 1 reply
  • 7 views

Kaito
Forum|alt.badge.img+2
  • Participating Frequently

Can Vertica get the data from MapR Streams using Vertica's kafka-related components directly ?

Thanks in advance?

Kaito Tonooka

1 reply

twall
Forum|alt.badge.img
  • Participating Frequently
  • July 6, 2017

Hi Kaito

I don't think this will work. While there is some compatibility between kafka & mapr streams, Vertica uses the low-level kafka consumer C API, and that not supported by mapr streams according to http://maprdocs.mapr.com/home/MapR_Streams/MapRStreamsCApplications.html.

Beyond that, I see there are some other differences in semantics, such as the way offsets are managed (mapr streams starts at 1 vs 0 in kafka) and the way security is done. Those may not be deal breakers for you, but those differences would probably break some scheduler use cases.

Tom