Skip to main content

Hadoop connector

  • May 2, 2014
  • 2 replies
  • 19 views

Muthu_Sundaram_1
Hadoop Connector - I believe we install this in every node of hadoop cluster?
What do we install in vertica cluster? What runs in vertica nodes to communicate with counterparts in hadoop nodes? What protocol does it use? Please provide more details regarding the communication between hadoop nodes and vertica nodes.

2 replies

R_McKeon
Forum|alt.badge.img
  • Participating Frequently
  • May 2, 2014
Hello Muthu,

Yes, you install the Hadoop Connector on each node. From https://my.vertica.com/docs/6.1.x/HTML/index.htm#15504.htm:

The Hadoop Connector runs on each node in the Hadoop cluster, so the Hadoop nodes and HP Vertica nodes communicate with each other directly. Direct connections allow data to be transferred in parallel, dramatically increasing processing speed.

The information regarding how Hadoop and Vertica communicate is located at the following links:

https://my.vertica.com/docs/6.1.x/HTML/index.htm#15503.htm
https://my.vertica.com/docs/6.1.x/HTML/index.htm#15511.htm

Thanks,
Rory


Muthu_Sundaram_1
Thanks Roy. I am trying to put an architectural diagram. Hadoop connectors are installed in Hadoop nodes. What is listening in Vertica clusters? What is the counterpart in Vertica side? Are hadoop clusters pushing data to Vertica using JDBC? What kind of protocol does it use? Should number of nodes between hadoop and Vertica match? Node1 from hadoop will talk directly to Node1 in vertica? Do we need any loadbalancer in vertica side for this?