Skip to main content

Transfer closed with data remaining to read when using HDFS connector

  • December 8, 2013
  • 1 reply
  • 11 views

Andy_He
Hi,

When copy data from HDFS to vertica using the standard HDFS connector, we encounter unexpected errors below:

COPY store_sales SOURCE Hdfs(url='http://ip-10-137-3-180.ec2.internal:50070/webhdfs/v1/user/root/relational_data/store_sales/*.dat', username='root');
ERROR 0:  Error calling process() in User Function UDSource at [src/Hdfs.cpp:105], error code: 0, message: [transfer closed with 662920330 bytes remaining to read]

What could be the problem? Thanks in advance.

1 reply

Forum|alt.badge.img+2
  • Participating Frequently
  • December 9, 2013
Hi Andy,

At first glance, it sounds like your HDFS server is closing the connection before the transfer is complete.

Are there any relevant messages in your HDFS log?

Adam