VerticaCopyStream processing speed
We use vertica 7 community edition on 20cores 32GB physical server. When we run VerticaCopyStream from our java program stream of 6.5MB (42k rows) is processed in 42seconds. CPU and disk utilisation is about 1%. How to improve it? I need to import handreds of GB. COPY #tmp_table_name FROM LOCAL STDIN DELIMITER '|' DIRECT ENFORCELENGTH We import the data to tmp table and then merge the data to other tables. BTW merge is fast enough , it takes few milliseconds.
+2Hi ,
How much it take when you try to load the same file via simple copy command from vsql ? do your java process run on the db server or on some other server , if on other server what is the CPU and memory utilization you see their ?
One of the way to scale data load is to run parallel load streams on different nodes .
We run our java app on different server. Import via vsql on the same node takes milliseconds. It's network problem. Thank you for the help.
Sign up
Already have an account? Login
Welcome to the Rocket Forum!
Please log in or register:
Employee Login | Registration Member Login | RegistrationEnter your E-mail address. We'll send you an e-mail with instructions to reset your password.
77 4th Avenue
Waltham, MA 02451 USA
