Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Creating compressed data with scooop

Copy link to this message
Re: Creating compressed data with scooop
I do not see any discrepancy here nor in the TaskTracker configuration. The mapreduce output compression should be working. Property:

* io.compression.codecs - Contains codes you were using
* mapred.output.compress - is set to true and is not set to final in TT config
* mapred.output.compression.codec - seems to be set correctly

Are you sure that your TaskTrackers were started with this configuration?

Would you mind sharing the map task output? Just to be sure that there  won't be anything suspicious.


On Tue, Jan 08, 2013 at 04:01:35PM +0800, Santosh Achhra wrote:
> Thank you Jarcec,
> I am posting values for few rows, Are you looking for any specific row ?
> Thanks once again for your help.
> mapred.compress.map.output true
> io.seqfile.lazydecompress true
> io.compression.codecs
> org.apache.hadoop.io.compress.DefaultCodec,org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.BZip2Codec,org.apache.hadoop.io.compress.DeflateCodec,org.apache.hadoop.io.compress.SnappyCodec
> mapred.output.compression.type BLOCK
> mapred.output.compression.codec org.apache.hadoop.io.compress.SnappyCodec
> io.seqfile.compress.blocksize 1000000
> dfs.image.compress false
> mapred.output.compress true
> dfs.image.compression.codec org.apache.hadoop.io.compress.DefaultCodec
> Good wishes,always !
> Santosh
> On Tue, Jan 8, 2013 at 3:36 PM, Jarek Jarcec Cecho <[EMAIL PROTECTED]>wrote:
> > L from JobTracker web ui by clicking at you job details (column