Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Is combiner and map in same JVM?


Copy link to this message
-
Re: Is combiner and map in same JVM?
Thanks. I am using 0.19, and to confirm, the map and combiner (in the map
jvm) are run in *different* threads at the same time?
My native library is not thread safe, so I would have to implement locks.
Aaron's email gave me hope(since the map and combiner would then be running
sequentially), but this appears to make things complicated.
Saptarshi Guha
On Tue, Apr 14, 2009 at 2:01 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote:

>
> On Apr 14, 2009, at 10:52 AM, Aaron Kimball wrote:
>
>  They're in the same JVM, and I believe in the same thread.
>>
>
> They are the same JVM. They *used* to be the same thread. In either 0.19 or
> 0.20, combiners are also called in the reduce JVM if spills are required.
>
> -- Owen
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB