Vertica-Spark connector: ipMap option

Question

Hey,
I'm trying to read data from Vertica into Spark using the Vertica-Spark connector using the following format:
spark.read.format("com.vertica.spark.datasource.DefaultSource").options(opts).load()

Problem is that it can't reach the private IPs that Vertica returns and then I found out about the "ipMap" param that allows me exactly to do the needed mapping to reach the public IPs.
I only saw this could be used when using the create() function but in the following post I saw that somebody actually used it inside the "opts" parameter which would solve my problem.https://forum.vertica.com/discussion/235065/spark

Does anybody here know in which String format I could do this? ("opts" must be Map[String, String])
Or if the example in the post is impossible and the only way for me to use ipMap is using the create() method?

Thanks a lot!

LenoyJ · Answer

Try settings export addresses as in this doc: https://www.vertica.com/kb/Configuring-Network-to-Import-and-Export-Data/Content/BestPractices/Configuring-Network-to-Import-and-Export-Data.htm.

Or, if you want to use ipmap, try something like the following in Opts:

       "ipmap" -> "192.168.61.77->172.16.61.77, 192.168.61.78->172.16.61.78, 192.168.61.79->172.16.61.79"

Where 192.168.61.77, 192.168.61.78, 192.168.61.79 are my private IPs and 172.16.61.77, 172.16.61.78, 172.16.61.79 are my public IPs.

Sign up

Please log in or register:

Welcome to the Rocket Forum!

Please log in or register: