Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Creating compressed data with scooop


Copy link to this message
-
Creating compressed data with scooop
Hello,

I am trying to import data from table, and I would like final data to be
compressed on HDFS which would help save some space.

I am executing below mentioned command.
This command completes successfully and I dont see any error reported
however when see the final data using  hadoop ls command is not in
compressed format in HDFS

sqoop --options-file /export/home/sqoop/connect.parm --table TEST
 --split-by F1  --compression-codec
org.apache.hadoop.io.compress.SnappyCodec -z

Am I missing something ?

Also I would like to know if I can import data into hive in compressed
format. I executed below mentioned command, in this case to data into HDFS
is not in compressed format and describe table command in hive  says that
table is not compressed

sqoop --options-file /export/home/sqoop/connect.parm --table TEST
 --split-by F1  --hive-import   -m 1  --compress --compression-codec
org.apache.hadoop.io.compress.GzipCodec

Good wishes,always !
Santosh
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB