I’ve questions from my customer about data loading from Hadoop. With “copy” command to reading many small data files directly from HDFS using “*” for loading all files in HDFS directory, Does Vertica load data in parallel on every node? What is the mechanism of our HDFS connector? If customer want to separate files into different directory and running many “copy” command, will it be faster? Thank you.

Question

Copy data from Hadoop - Parallel or Sequential

I’ve questions from my customer about data loading from Hadoop.

With “copy” command to reading many small data files directly from HDFS using “*” for loading all files in HDFS directory, Does Vertica load data in parallel on every node?
What is the mechanism of our HDFS connector?
If customer want to separate files into different directory and running many “copy” command, will it be faster?

Thank you.