Environment
Vertica Analytics Database: v25.x
Situation
The native Vertica cluster health check shows the catalog truncat lag. Upon trying to sync catalog, it is learnt the nodes are failing to authorize with IAM roles in their attempt to communicate with communal storage (AWS S3 bucket). The nodes have been configured to use IAM access to communicate with S3 bucket. Below is the error the customer sees when attempting a catalog sync -
[Code: 8076, SQL State: VX001] [Vertica][VJDBC](8076) ERROR: Could not copy file [/vertica/catalog/XXX_Database_name/v_xxx_database_name_node0004_catalog/Catalog/Txnlogs/txn_29326157_g14830882.cat] to [s3://communal-storage-location/xxx_database_name/metadata/xxx_database_name/nodes/v_xxx_database_name_prod_node0004/Catalog/9731c35e9fd39349bcd0b77ca82795/Txnlogs/txn_29326157_g14830882.cat]: Unable to parse ExceptionName: ExpiredToken Message: The provided token has expired.
Upon restarting the Vertica service on the impacted node, the catalog sync is successful and the errors disappears.