Skip to main content

Slow performance for COPY LOCAL command, it uses only one machine in the cluster!

  • January 5, 2016
  • 3 replies
  • 14 views

Ammar

Hi,

 

I am copying data, from local 1000 files with total size about 1.7 GB, to a three-nodes Vertica cluster. I use this command:

 

 

COPY toy_edge from LOCAL '/home/dbadmin/toyGraph*'  DELIMITER ' ';

I created the database on all nodes using :

 

 

admintools -t create_db -s ip1,ip2,ip3 -d graphDB

 

The process took long time. It is been almost 1 hour and the data is not loaded yet. Moreover, I noticed that only machine ip1 is busy (CPU=100%), the other two machines are idle. 

 

Kindly note that the file exist on the machine ip1, however, I expected vertica to transfer some data to be stored to other machines as well.

 

Thanks,

-Khaled

 

 

 

3 replies

eli_revach
Forum|alt.badge.img+2
  • Participating Frequently
  • January 5, 2016

Hi ,

To  have parallel load you need to put your files on an NFS where each node can access  or splite the files manulay to each node on the cluster .

 

Thanks 


Ammar
  • Author
  • New Participant
  • January 16, 2016

Can I use HDFS instead od NFS ? How?

 

Thanks,

-Khaled

 


eli_revach
Forum|alt.badge.img+2
  • Participating Frequently
  • January 17, 2016