Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Creating compressed data with scooop

Copy link to this message
Creating compressed data with scooop

I am trying to import data from table, and I would like final data to be
compressed on HDFS which would help save some space.

I am executing below mentioned command.
This command completes successfully and I dont see any error reported
however when see the final data using  hadoop ls command is not in
compressed format in HDFS

sqoop --options-file /export/home/sqoop/connect.parm --table TEST
 --split-by F1  --compression-codec
org.apache.hadoop.io.compress.SnappyCodec -z

Am I missing something ?

Also I would like to know if I can import data into hive in compressed
format. I executed below mentioned command, in this case to data into HDFS
is not in compressed format and describe table command in hive  says that
table is not compressed

sqoop --options-file /export/home/sqoop/connect.parm --table TEST
 --split-by F1  --hive-import   -m 1  --compress --compression-codec

Good wishes,always !