Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> data in compression format affect mapreduce speed


+
shangan 2010-08-26, 02:32
Copy link to this message
-
Re: data in compression format affect mapreduce speed
Compressed data would increase processing time in mapper/reducer but
decrease the amount of data transferred between tasktracker nodes.
Normally you should consider applying some form of compression.

On Wed, Aug 25, 2010 at 7:32 PM, shangan <[EMAIL PROTECTED]> wrote:

> will data stored in  compression format affect mapreduce job speed?
> increase or decrease? or more complex relationship between these two ?  can
> anybody give some explanation in detail?
>
> 2010-08-26
>
>
>
> shangan
>
+
Harsh J 2010-08-26, 04:14
+
shangan 2010-08-26, 05:50
+
Harsh J 2010-08-26, 06:08
+
Greg Roelofs 2010-08-26, 19:57
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB