Skip to main content
Question

Is it possible to access AWS S3 on different AWS account with IAM roles on Vertica machine?

  • January 7, 2022
  • 6 replies
  • 13 views
Poslanik

Hi,
We have been using Vertica 9.3 version hosted on AWS EC2 instances on our AWS account A and were able to access S3 bucket on another AWS account B with Bucket policy on that S3 bucket with Principal Role which is assigned to EC2 instances where Vertica is deployed on AWS account A.
AWS account A AWS account B
Vertica 9.3 on EC2 instance with AWS Vertica-role S3 bucket policy Principal Vertica-role

After upgrading to Vertica version 11 we are not able to reach AWS account B S3 bucket with the same setup.
We are told to use ALTER SESSION SET AWSAuth.
For that on AWS account B on S3 bucket we need to have Principal AWS user (instead of a role) and in our SQL queries we need to hardcode accesskey and secretaccesskey of that new AWS user.

ALTER SESSION SET AWSAuth='XXXXXX:XXXXXXXXXXXXXXX';

Vertica documentation says we should use AWS IAM roles to access AWS resources, but we are not able to.

Please tell me there is a way and that I am missing something.

Thank you

    6 replies

    SruthiA
    Forum|alt.badge.img+1
    • SruthiA
    • Participating Frequently
    • January 17, 2022

    @Poslanik : Could you please share me the error message you are receiving?

    SruthiA
    Forum|alt.badge.img+1
    • SruthiA
    • Participating Frequently
    • February 21, 2022

    @Poslanik : Please set AWSEndpoint with the value pertaining to the region you are trying to use and try.. If you just set AWSAuth, is it succeeding?

    Poslanik
    • Poslanik
    • Author
    • Participating Frequently
    • February 23, 2022

    @SruthiA,
    If I just set AWSAuth without setting AWSRegion, it also fails with the same error.
    Probably because Vertica default AWS region and S3 bucket region from where I try to COPY data are different.

    For it to succeed I have to execute:

    ALTER SESSION SET AWSRegion='eu-west-1';
ALTER SESSION SET AWSAuth='************:***********************';
COPY <table_name>
(<list_of_columns>)
from
's3://<some_file_path>.json.gz'
FILTER GZip() parser fjsonparser(flatten_maps = false) direct;

    Could you please elaborate what do you mean by > Please set AWSEndpoint with the value pertaining to the region you are trying to use
    I see AWSEndpoint parameter has default value s3.amazonaws.com, so not sure what I need to set.

    Thank you

    SruthiA
    Forum|alt.badge.img+1
    • SruthiA
    • Participating Frequently
    • February 23, 2022

    @Poslanik : please find the below link which contains endpoint URLS for all the regions. Please set it according your region and retry.

    https://docs.aws.amazon.com/general/latest/gr/s3.html

    Poslanik
    • Poslanik
    • Author
    • Participating Frequently
    • February 24, 2022

    Unfortunately I am getting the same error after I set AWSEndpoint to the region of a S3 bucket from where I am trying to COPY data to Vertica.

    ALTER SESSION SET AWSEndpoint='s3.eu-west-1.amazonaws.com';

    Also tried with other standard endpoints.

    SruthiA
    Forum|alt.badge.img+1
    • SruthiA
    • Participating Frequently
    • February 24, 2022

    @Poslanik : Could you please restart the cluster and try once?

    Still scrolling?
    Ask a question
    Terms & ConditionsAccessibility statement