Skip to main content

Copy lzo compressed file from hdfs to Vertica using vertica hdfs connector

  • January 21, 2015
  • 2 replies
  • 18 views

Amit_1
Forum|alt.badge.img+1
  • Participating Frequently
Can we copy lzo compressed file from hdfs to Vertica using vertica hdfs connector ?

I tried the below but looks like its not able to identify LZO

Copy abc.test Hdfs(url='http://prd101.corp.test.net:40070/webhdfs/v1/vertica/data/lzo/*', username='xyz') LZO DELIMITER ',' NULL AS '' ESCAPE AS E'\001';

ERROR 4856:  Syntax error at or near "LZO" at character 180

LINE 1: .../vertica/data/lzo/*', username='sys_vertica_sbm') LZO DELIMI...

                                                             ^

The below page says we can do it using Copy command but not sure if thats possible for Copy with hdfs connector?

If not is there any other way we can copy lzo files from hdfs to vertica?

Regards,
-Amit

2 replies

Rahul_Choudhary
Forum|alt.badge.img
  • Participating Frequently
  • January 21, 2015
Hi Amit

We do not currently support loading data compressed in LZO on HDFS. A feature request has been submitted to have this feature added to a future release of the product. 


Regards
Rahul Choudhary

soumyaranjan
  • New Participant
  • September 7, 2015

Hi Amit/Rahul,
     Do we have any solution for the same now ? Or any alternative way to do the same. 

Thanks,
Soumyaranjan