Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Sqoop >> mail # user >> Creating compressed data with scooop


Copy link to this message
-
Creating compressed data with scooop
Hello,

I am trying to import data from table, and I would like final data to be
compressed on HDFS which would help save some space.

I am executing below mentioned command.
This command completes successfully and I dont see any error reported
however when see the final data using  hadoop ls command is not in
compressed format in HDFS

sqoop --options-file /export/home/sqoop/connect.parm --table TEST
 --split-by F1  --compression-codec
org.apache.hadoop.io.compress.SnappyCodec -z

Am I missing something ?

Also I would like to know if I can import data into hive in compressed
format. I executed below mentioned command, in this case to data into HDFS
is not in compressed format and describe table command in hive  says that
table is not compressed

sqoop --options-file /export/home/sqoop/connect.parm --table TEST
 --split-by F1  --hive-import   -m 1  --compress --compression-codec
org.apache.hadoop.io.compress.GzipCodec

Good wishes,always !
Santosh
+
Jarek Jarcec Cecho 2013-01-07, 12:31
+
Santosh Achhra 2013-01-08, 05:21
+
Jarek Jarcec Cecho 2013-01-08, 07:36
+
Santosh Achhra 2013-01-08, 08:01
+
Jarek Jarcec Cecho 2013-01-08, 11:09
+
Santosh Achhra 2013-01-08, 14:33
+
Jarek Jarcec Cecho 2013-01-08, 15:43
+
Santosh Achhra 2013-01-08, 16:09
+
Jarek Jarcec Cecho 2013-01-09, 10:01
+
Santosh Achhra 2013-01-09, 11:45
+
Jarek Jarcec Cecho 2013-01-10, 09:40
+
Santosh Achhra 2013-01-10, 12:47