Skip to main content

Vertica Spark Connector 0.2.0- Invalid Projection issue

  • Mar 8, 2016 10:48 AM UTC
  • 3 replies
  • 32 views

lciszak

I'm testing the Vertica Spark Connector 0.2.0 with Vertica 7.2.1 and HDP 2.4.0 (Spark 1.6).

I'm going through the examples from the user guide and they seem to work fine only when the tables from which I'm extracting the data are in the default schema.

However, if the table is not in the default schema and I add the parameter "dbschema"  to the options, I get the following exception:

 

java.sql.SQLSyntaxErrorException: [Vertica][VJDBC](3737) ERROR: Invalid projection name category_b0

 

The examples in the User guide somehow never provide the dbschema parameter, so I'm wondering if this behavior is intentional.

 

Here's the code I'm using:

 

val hostName = "..."
val db = "dsa"
val tableName1 = "category"
val part = "12";
val dbschema = "ga_master"
val opt = Map("host" -> hostName, "table" -> tableName1, "db" -> db, "numPartitions" -> part, "user" -> "...", "password" -> "...", "dbschema" -> dbschema)
val df = sqlContext.read.format("com.vertica.spark.datasource.DefaultSource").options(opt).load()

 

3 replies

Harshad
Forum|alt.badge.img
  • Participating Frequently
  • Mar 9, 2016 9:42 PM UTC

Hi lciszak,

 

I tested this in my lab, and was able to reproduce the issue. It only worked with a default schema.

I checked this internally with the relevant folks; the fix for this issue is currently in the works.

 

Thanks,

Harshad


EdwardM
Forum|alt.badge.img
  • Participating Frequently
  • Mar 15, 2016 7:49 PM UTC

Hey guys,

 

Please try v0.2.1, which is now available. This release should fix this issue.

 

Thanks,

Edward


lciszak
  • Author
  • New Participant
  • Mar 15, 2016 9:36 PM UTC

Good stuff! Thanks guys.