Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> data in compression format affect mapreduce speed


Copy link to this message
-
Re: data in compression format affect mapreduce speed
Compressed data would increase processing time in mapper/reducer but
decrease the amount of data transferred between tasktracker nodes.
Normally you should consider applying some form of compression.

On Wed, Aug 25, 2010 at 7:32 PM, shangan <[EMAIL PROTECTED]> wrote:

> will data stored in  compression format affect mapreduce job speed?
> increase or decrease? or more complex relationship between these two ?  can
> anybody give some explanation in detail?
>
> 2010-08-26
>
>
>
> shangan
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB