Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Is combiner and map in same JVM?


Copy link to this message
-
Re: Is combiner and map in same JVM?
Saptarshi Guha 2009-04-14, 18:10
Thanks. I am using 0.19, and to confirm, the map and combiner (in the map
jvm) are run in *different* threads at the same time?
My native library is not thread safe, so I would have to implement locks.
Aaron's email gave me hope(since the map and combiner would then be running
sequentially), but this appears to make things complicated.
Saptarshi Guha
On Tue, Apr 14, 2009 at 2:01 PM, Owen O'Malley <[EMAIL PROTECTED]> wrote:

>
> On Apr 14, 2009, at 10:52 AM, Aaron Kimball wrote:
>
>  They're in the same JVM, and I believe in the same thread.
>>
>
> They are the same JVM. They *used* to be the same thread. In either 0.19 or
> 0.20, combiners are also called in the reduce JVM if spills are required.
>
> -- Owen
>