Skip to main content

Importing from Amazon S3 using the Vertica Library for Amazon Web Services

  • June 13, 2018
  • 5 replies
  • 15 views
ishangaur

Hi,
I have a use case where i am trying to load the data into one of the Vertica table from a .csv file present in AWS S3 bucket.

I would like to know how can i configure the Vertica Library for Amazon Web Services Parameters ?

I did try doing it using the instructions provided on the below link , but it did not work -->

https://my.vertica.com/docs/7.2.x/HTML/index.htm#Authoring/AdministratorsGuide/ConfiguringTheDB/AwsLibraryParameters.htm

AFTER completing the steps above i ran the below command , but getting the error -->

ERROR MESSAGE - ERROR 6085: Unexpected exception calling process() User Function in UDSource

All i need to achieve is load the data to vertica table from s3 . command that i used is as below -

.

<

table name> WITH SOURCE S3(url='s3://idl-sbg-uw2-processed-analytics-prod-prd/<filename.csv>');

If anyone has done this kind of usecase , please let me know . Appreciate your help.

    5 replies

    asaeidi
    Forum|alt.badge.img
    • asaeidi
    • Participating Frequently
    • June 13, 2018

    Hi, what's version of Vertica you're using?

    ishangaur
    • ishangaur
    • Author
    • New Participant
    • June 13, 2018

    Vertica Analytic Database v8.1.1-15

    ishangaur
    • ishangaur
    • Author
    • New Participant
    • June 13, 2018

    I tried executing these commands on VSQL first and then copy command as below -- >

    ALTER SESSION SET UDPARAMETER FOR awslib aws_region='us-west-2';
    ALTER SESSION SET UDPARAMETER FOR awslib aws_id='xxx';
    ALTER SESSION SET UDPARAMETER FOR awslib aws_secret='XXX';

    => copy abc.table SOURCE S3(url='s3://s3bucket/xyz.csv') DELIMITER ',';
    ERROR 5861: Error calling process() in User Function UDSource at [src/S3.cpp:137], error code: 0, message: Client Error: The AWS Access Key Id you provided does not exist in our records

    ishangaur
    • ishangaur
    • Author
    • New Participant
    • June 13, 2018

    Primarily i want to know how do i configure my vertica or what kind of configuration needs to be done in order to enable this copy from S3 to vertica.

    Jim_Knicely
    Forum|alt.badge.img+2
    • Jim_Knicely
    • Participating Frequently
    • June 13, 2018

    Looks like you are attempting to configure it properly. The error "The AWS Access Key Id you provided does not exist in our records" is coming from AWS, not Vertica.

    I think you are using an invalid AWS_ID.

    Still scrolling?
    Ask a question
    Terms & ConditionsAccessibility statement