Significant performance difference between COPY LOCAL and COPY FROM STDIN

I see much worse performance when using STDIN.

[dbadmin@SE-Sandbox-26-node1 ~]$ vsql -c "\d big_local_copy;"
                                        List of Fields by Tables
 Schema |     Table      | Column |    Type     | Size | Default | Not Null | Primary Key | Foreign Key
--------+----------------+--------+-------------+------+---------+----------+-------------+-------------
 public | big_local_copy | c1     | int         |    8 |         | f        | f           |
 public | big_local_copy | c2     | varchar(10) |   10 |         | f        | f           |
 public | big_local_copy | c3     | varchar(10) |   10 |         | f        | f           |
(3 rows)

[dbadmin@SE-Sandbox-26-node1 ~]$ wc -l big.txt
8871936 big.txt

[dbadmin@SE-Sandbox-26-node1 ~]$ vsql -ic "COPY big_local_copy FROM LOCAL '/home/dbadmin/big.txt' DIRECT;"
 Rows Loaded
-------------
     8871936
(1 row)

Time: First fetch (1 row): 7459.585 ms. All rows formatted: 7459.714 ms

[dbadmin@SE-Sandbox-26-node1 ~]$ vsql -ic "COPY big_local_copy FROM STDIN DIRECT;" < /home/dbadmin/big.txt
Time: First fetch (0 rows): 24207.292 ms. All rows formatted: 24207.300 ms

It may be file size related. My test file is 20G, comparable to what the customer loads in each batch.

Here is a test using a 25 GB file. It was about 14 minutes faster to use COPY LOCAL. Note that I am using Vertica 9.3.1-0 on three nodes.

[dbadmin@SE-Sandbox-26-node1 ~]$ ls -lrth /home/dbadmin/bigger.txt
-rw-r--r--. 1 dbadmin verticadba 25G Jan 15 15:51 /home/dbadmin/bigger.txt

[dbadmin@SE-Sandbox-26-node1 ~]$ vsql -c "\d bigger;"
                                    List of Fields by Tables
 Schema | Table  | Column |    Type     | Size | Default | Not Null | Primary Key | Foreign Key
--------+--------+--------+-------------+------+---------+----------+-------------+-------------
 public | bigger | c1     | varchar(80) |   80 |         | f        | f           |
 public | bigger | c2     | varchar(80) |   80 |         | f        | f           |
 public | bigger | c3     | varchar(80) |   80 |         | f        | f           |
 public | bigger | c4     | varchar(80) |   80 |         | f        | f           |
 public | bigger | c5     | varchar(80) |   80 |         | f        | f           |
 public | bigger | c6     | varchar(80) |   80 |         | f        | f           |
 public | bigger | c7     | varchar(80) |   80 |         | f        | f           |
(7 rows)

[dbadmin@SE-Sandbox-26-node1 ~]$ vsql -ic "COPY bigger FROM LOCAL '/home/dbadmin/bigger.txt' DIRECT;"
 Rows Loaded
-------------
   637880912
(1 row)

Time: First fetch (1 row): 1045943.399 ms. All rows formatted: 1045943.915 ms

[dbadmin@SE-Sandbox-26-node1 ~]$ vsql -c "TRUNCATE TABLE bigger;"
TRUNCATE TABLE

[dbadmin@SE-Sandbox-26-node1 ~]$ vsql -ic "COPY bigger FROM STDIN DIRECT;" < /home/dbadmin/bigger.txt
Time: First fetch (0 rows): 1860550.277 ms. All rows formatted: 1860550.289 ms