Skip to main content

copy s3 ( .csv file Data ) from AWS s3 to vertica table

  • June 13, 2018
  • 1 reply
  • 7 views
ishangaur

I am unable to copy the data from the .csv file present on AWS s3 to vertica .
I am executing these commands on VSQL first and then copy command as below -- >

ALTER SESSION SET UDPARAMETER FOR awslib aws_region='us-west-2';
 ALTER SESSION SET UDPARAMETER FOR awslib aws_id='xxx';
ALTER SESSION SET UDPARAMETER FOR awslib aws_secret='XXX';

=> copy abc.table SOURCE S3(url='s3://s3bucket/xyz.csv') DELIMITER ',';
ERROR 5861:  Error calling process() in User Function UDSource at [src/S3.cpp:137], error code: 0, message: Client Error: The AWS Access Key Id you provided does not exist in our records.

Please help.

    1 reply

    Jim_Knicely
    Forum|alt.badge.img+2
    • Jim_Knicely
    • Participating Frequently
    • June 13, 2018

    Looks like you are attempting to configure it properly. The error "The AWS Access Key Id you provided does not exist in our records" is coming from AWS, not Vertica.

    I think you are using an invalid AWS_ID.

    I can easily duplicate the error...

    dbadmin=> ALTER SESSION SET UDPARAMETER FOR awslib aws_id='... CORRECT ID ...';
ALTER SESSION

dbadmin=> ALTER SESSION SET UDPARAMETER FOR awslib aws_secret='... CORRECT SECRET ...;
ALTER SESSION

dbadmin=> ALTER SESSION SET UDPARAMETER FOR awslib aws_region='us-east-2';
ALTER SESSION

dbadmin=> CREATE FLEX EXTERNAL TABLE ge() AS COPY SOURCE s3(bucket='s3://vertica-mybucket/*.json') PARSER fjsonparser();
CREATE TABLE

dbadmin=> SELECT name, type, height, hike_safety FROM ge;
      name      |   type   | height | hike_safety
----------------+----------+--------+-------------
 Arenal Volcano | volcano  | 5479   | 12.4
 Joshua         | mountain | 236172 | 100
 Everest        | mountain | 29029  | 34.1
 Mt St Helens   | volcano  | 29029  | 15.4
 Denali         | mountain | 17000  | 12.2
 Kilimanjaro    | mountain | 14000  |
 Mt Washington  | mountain |        | 50.6
(7 rows)

    Now I will change to an invalid AWS_ID:

    dbadmin=> ALTER SESSION SET UDPARAMETER FOR awslib aws_id='... INCORRECT ID ...';
ALTER SESSION

dbadmin=> SELECT name, type, height, hike_safety FROM ge;
ERROR 5861:  Error calling setup() in User Function UDSource at [/data/build-centos6/qb/buildagent/workspace/jenkins2/ReleaseBuilds/Grader/REL-9_1_0-x_grader/build/udx/supported/AWS/../../../vertica/SAL/S3UDFS/S3Helpers_internal.cpp:894], error code: 5, message: The AWS Access Key Id you provided does not exist in our records.

    I got the same error as you!

    Still scrolling?
    Ask a question
    Terms & ConditionsAccessibility statement