Skip to main content

where to find COPY error files in EON mode

  • December 4, 2017
  • 5 replies
  • 15 views

DieterC
Forum|alt.badge.img+2

Hi forum,
we have started our first data COPYs to an EON DB. Can you advise where the error files are located?
I have found a directory metadata/dbname/nodes/nodename/Catalog directory with to subdirs, Checkpoints and Txnlogs.
Is this the place to look for?
Please advise.
Thanks
Dieter

5 replies

skeswani
Forum|alt.badge.img
  • Participating Frequently
  • December 4, 2017

There should be a directory CopyErrorLogs in the catalog directory.
Can you also explicitly specify the file name or table name in the copy command.
see copy command syntax for that


DieterC
Forum|alt.badge.img+2
  • Author
  • Participating Frequently
  • December 4, 2017

The error logs are written to S3, not the DB node, correct?
The metadata/dbname/nodes/nodename/Catalog in the S3 bucket (ROS location) is the catalog location?
Thanks


Jim_Knicely
Forum|alt.badge.img+2
  • Participating Frequently
  • December 4, 2017

Check the nodes system table for the catalog path:

Example:

dbadmin=> select catalog_path from nodes;
                    catalog_path
----------------------------------------------------
 /home/dbadmin/sfdc/v_sfdc_node0001_catalog/Catalog
(1 row)

As @skeswani said, there should be a CopyErrorLogs directory in the catalog directory.

Example:

[dbadmin@s18384357 v_sfdc_node0001_catalog]$ ls /home/dbadmin/sfdc/v_sfdc_node0001_catalog/CopyErrorLogs | head -2
customers-data.txt-copy-from-exceptions
customers-data.txt-copy-from-rejected-data

Would it be easier to use the "rejected data table" functionality of the COPY command?


DieterC
Forum|alt.badge.img+2
  • Author
  • Participating Frequently
  • December 4, 2017

Thanks Jim, good to know the catalog is on the V node, NOT in S3.


Ben_Vandiver
Forum|alt.badge.img
  • Participating Frequently
  • December 4, 2017

We have a local copy of the catalog on the V node. There's also a copy in S3. But the error logs won't end up in S3 unless you ask for them to be written there. I would try
COPY foo FROM 's3://mybucket/data/*' REJECTED DATA 's3://mybucket/loaderrors/';
or (even cooler!) REJECTED DATA AS TABLE foo_errors;